cpython-lru_cache-weakref: Objects/unicodeobject.c annotate

annotate Objects/unicodeobject.c @ 41794:2c00a1910271 legacy-trunk

Move imgfile import to the global namespace to trigger an import error ASAP to prevent creation of a test file.

author	Brett Cannon <bcannon@gmail.com>
date	Sun, 20 May 2007 23:56:18 +0000
parents	d01bd37bf5ce
children	e62b606eff0b

rev	line source
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	1 /*
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3 Unicode implementation based on original code by Fredrik Lundh,
12646 17d1a3f1f085 Minimal change so I can add the rest of MAL's checkin message: Fred Drake <fdrake@acm.org> parents: 12645 diff changeset	4 modified by Marc-Andre Lemburg <mal@lemburg.com> according to the
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5 Unicode Integration Proposal (see file Misc/unicode.txt).
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6
38335 f9332ad4287a needforspeed: partition implementation, part two. Fredrik Lundh <fredrik@pythonware.com> parents: 38304 diff changeset	7 Major speed upgrades to the method implementations at the Reykjavik
f9332ad4287a needforspeed: partition implementation, part two. Fredrik Lundh <fredrik@pythonware.com> parents: 38304 diff changeset	8 NeedForSpeed sprint, by Fredrik Lundh and Andrew Dalke.
f9332ad4287a needforspeed: partition implementation, part two. Fredrik Lundh <fredrik@pythonware.com> parents: 38304 diff changeset	9
13828 254f8b7c4a83 Changing the CNRI copyright notice according to CNRI's instructions. Guido van Rossum <guido@python.org> parents: 13761 diff changeset	10 Copyright (c) Corporation for National Research Initiatives.
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	11
16029 55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	12 --------------------------------------------------------------------
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	13 The original string type implementation is:
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	14
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	15 Copyright (c) 1999 by Secret Labs AB
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	16 Copyright (c) 1999 by Fredrik Lundh
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	17
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	18 By obtaining, using, and/or copying this software and/or its
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	19 associated documentation, you agree that you have read, understood,
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	20 and will comply with the following terms and conditions:
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	21
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	22 Permission to use, copy, modify, and distribute this software and its
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	23 associated documentation for any purpose and without fee is hereby
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	24 granted, provided that the above copyright notice appears in all
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	25 copies, and that both that copyright notice and this permission notice
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	26 appear in supporting documentation, and that the name of Secret Labs
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	27 AB or the author not be used in advertising or publicity pertaining to
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	28 distribution of the software without specific, written prior
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	29 permission.
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	30
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	31 SECRET LABS AB AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD TO
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	32 THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	33 FITNESS. IN NO EVENT SHALL SECRET LABS AB OR THE AUTHOR BE LIABLE FOR
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	34 ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	35 WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	36 ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	37 OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	38 --------------------------------------------------------------------
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	39
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	40 */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	41
37702 9b22d8b7bcaa Make Py_BuildValue, PyObject_CallFunction and Martin v. Löwis <martin@v.loewis.de> parents: 37668 diff changeset	42 #define PY_SSIZE_T_CLEAN
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	43 #include "Python.h"
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	44
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	45 #include "unicodeobject.h"
13209 b11e0a898de0 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13196 diff changeset	46 #include "ucnhash.h"
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	47
24071 8142f8d668a0 Patch #569753: Remove support for WIN16. Martin v. Löwis <martin@v.loewis.de> parents: 23837 diff changeset	48 #ifdef MS_WINDOWS
12067 bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	49 #include <windows.h>
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	50 #endif
12329 3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	51
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	52 /* Limit for the Unicode object free list */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	53
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	54 #define MAX_UNICODE_FREELIST_SIZE 1024
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	55
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	56 /* Limit for the Unicode object free list stay alive optimization.
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	57
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	58 The implementation will keep allocated Unicode memory intact for
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	59 all objects on the free list having a size less than this
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	60 limit. This reduces malloc() overhead for small Unicode objects.
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	61
12025 7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said: Barry Warsaw <barry@python.org> parents: 11997 diff changeset	62 At worst this will result in MAX_UNICODE_FREELIST_SIZE *
12329 3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	63 (sizeof(PyUnicodeObject) + KEEPALIVE_SIZE_LIMIT +
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	64 malloc()-overhead) bytes of unused garbage.
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	65
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	66 Setting the limit to 0 effectively turns the feature off.
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	67
12329 3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	68 Note: This is an experimental feature ! If you get core dumps when
3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	69 using Unicode objects, turn this feature off.
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	70
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	71 */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	72
12329 3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	73 #define KEEPALIVE_SIZE_LIMIT 9
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	74
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	75 /* Endianness switches; defaults to little endian */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	76
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	77 #ifdef WORDS_BIGENDIAN
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	78 # define BYTEORDER_IS_BIG_ENDIAN
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	79 #else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	80 # define BYTEORDER_IS_LITTLE_ENDIAN
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	81 #endif
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	82
12899 9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	83 /* --- Globals ------------------------------------------------------------
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	84
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	85 The globals are initialized by the _PyUnicode_Init() API and should
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	86 not be used before calling that API.
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	87
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	88 */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	89
37655 2f3f2e32dfd9 spread the extern "C" { } magic pixie dust around. Python itself builds now Anthony Baxter <anthonybaxter@gmail.com> parents: 37596 diff changeset	90
2f3f2e32dfd9 spread the extern "C" { } magic pixie dust around. Python itself builds now Anthony Baxter <anthonybaxter@gmail.com> parents: 37596 diff changeset	91 #ifdef __cplusplus
2f3f2e32dfd9 spread the extern "C" { } magic pixie dust around. Python itself builds now Anthony Baxter <anthonybaxter@gmail.com> parents: 37596 diff changeset	92 extern "C" {
2f3f2e32dfd9 spread the extern "C" { } magic pixie dust around. Python itself builds now Anthony Baxter <anthonybaxter@gmail.com> parents: 37596 diff changeset	93 #endif
2f3f2e32dfd9 spread the extern "C" { } magic pixie dust around. Python itself builds now Anthony Baxter <anthonybaxter@gmail.com> parents: 37596 diff changeset	94
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	95 /* Free list for Unicode objects */
12899 9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	96 static PyUnicodeObject *unicode_freelist;
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	97 static int unicode_freelist_size;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	98
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	99 /* The empty Unicode object is shared to improve performance. */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	100 static PyUnicodeObject *unicode_empty;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	101
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	102 /* Single character Unicode strings in the Latin-1 range are being
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	103 shared as well. */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	104 static PyUnicodeObject *unicode_latin1[256];
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	105
12645 c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	106 /* Default encoding to use and assume when NULL is passed as encoding
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	107 parameter; it is initialized by _PyUnicode_Init().
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	108
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	109 Always use the PyUnicode_SetDefaultEncoding() and
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	110 PyUnicode_GetDefaultEncoding() APIs to access this global.
12645 c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	111
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	112 */
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	113 static char unicode_default_encoding[100];
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	114
18298 982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE. Martin v. Löwis <martin@v.loewis.de> parents: 18295 diff changeset	115 Py_UNICODE
18774 57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	116 PyUnicode_GetMax(void)
18298 982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE. Martin v. Löwis <martin@v.loewis.de> parents: 18295 diff changeset	117 {
18324 3ba7a358ce02 use Py_UNICODE_WIDE instead of USE_UCS4_STORAGE and Py_UNICODE_SIZE Fredrik Lundh <fredrik@pythonware.com> parents: 18298 diff changeset	118 #ifdef Py_UNICODE_WIDE
18298 982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE. Martin v. Löwis <martin@v.loewis.de> parents: 18295 diff changeset	119 return 0x10FFFF;
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE. Martin v. Löwis <martin@v.loewis.de> parents: 18295 diff changeset	120 #else
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE. Martin v. Löwis <martin@v.loewis.de> parents: 18295 diff changeset	121 /* This is actually an illegal character, so it should
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE. Martin v. Löwis <martin@v.loewis.de> parents: 18295 diff changeset	122 not be passed to unichr. */
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE. Martin v. Löwis <martin@v.loewis.de> parents: 18295 diff changeset	123 return 0xFFFF;
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE. Martin v. Löwis <martin@v.loewis.de> parents: 18295 diff changeset	124 #endif
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE. Martin v. Löwis <martin@v.loewis.de> parents: 18295 diff changeset	125 }
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE. Martin v. Löwis <martin@v.loewis.de> parents: 18295 diff changeset	126
38257 af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	127 /* --- Bloom Filters ----------------------------------------------------- */
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	128
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	129 /* stuff to implement simple "bloom filters" for Unicode characters.
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	130 to keep things simple, we use a single bitmask, using the least 5
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	131 bits from each unicode characters as the bit index. */
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	132
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	133 /* the linebreak mask is set up by Unicode_Init below */
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	134
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	135 #define BLOOM_MASK unsigned long
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	136
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	137 static BLOOM_MASK bloom_linebreak;
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	138
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	139 #define BLOOM(mask, ch) ((mask & (1 << ((ch) & 0x1F))))
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	140
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	141 #define BLOOM_LINEBREAK(ch)\
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	142 (BLOOM(bloom_linebreak, (ch)) && Py_UNICODE_ISLINEBREAK((ch)))
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	143
38440 5b0753b9d16a needforspeed: replace improvements, changed to Py_LOCAL_INLINE Fredrik Lundh <fredrik@pythonware.com> parents: 38427 diff changeset	144 Py_LOCAL_INLINE(BLOOM_MASK) make_bloom_mask(Py_UNICODE* ptr, Py_ssize_t len)
38257 af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	145 {
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	146 /* calculate simple bloom-style bitmask for a given unicode string */
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	147
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	148 long mask;
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	149 Py_ssize_t i;
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	150
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	151 mask = 0;
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	152 for (i = 0; i < len; i++)
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	153 mask \|= (1 << (ptr[i] & 0x1F));
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	154
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	155 return mask;
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	156 }
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	157
38440 5b0753b9d16a needforspeed: replace improvements, changed to Py_LOCAL_INLINE Fredrik Lundh <fredrik@pythonware.com> parents: 38427 diff changeset	158 Py_LOCAL_INLINE(int) unicode_member(Py_UNICODE chr, Py_UNICODE* set, Py_ssize_t setlen)
38257 af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	159 {
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	160 Py_ssize_t i;
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	161
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	162 for (i = 0; i < setlen; i++)
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	163 if (set[i] == chr)
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	164 return 1;
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	165
38270 1dc5657f1745 return 0 on misses, not -1. Fredrik Lundh <fredrik@pythonware.com> parents: 38257 diff changeset	166 return 0;
38257 af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	167 }
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	168
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	169 #define BLOOM_MEMBER(mask, chr, set, setlen)\
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	170 BLOOM(mask, chr) && unicode_member(chr, set, setlen)
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	171
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	172 /* --- Unicode Object ----------------------------------------------------- */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	173
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	174 static
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	175 int unicode_resize(register PyUnicodeObject *unicode,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	176 Py_ssize_t length)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	177 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	178 void *oldstr;
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	179
12329 3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	180 /* Shortcut if there's nothing much to do. */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	181 if (unicode->length == length)
12329 3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	182 goto reset;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	183
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	184 /* Resizing shared object (unicode_empty or single character
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	185 objects) in-place is not allowed. Use PyUnicode_Resize()
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	186 instead ! */
38335 f9332ad4287a needforspeed: partition implementation, part two. Fredrik Lundh <fredrik@pythonware.com> parents: 38304 diff changeset	187
37661 b1c6a3864b1e Revert 34153: Py_UNICODE should not be signed. Martin v. Löwis <martin@v.loewis.de> parents: 37655 diff changeset	188 if (unicode == unicode_empty \|\|
b1c6a3864b1e Revert 34153: Py_UNICODE should not be signed. Martin v. Löwis <martin@v.loewis.de> parents: 37655 diff changeset	189 (unicode->length == 1 &&
b1c6a3864b1e Revert 34153: Py_UNICODE should not be signed. Martin v. Löwis <martin@v.loewis.de> parents: 37655 diff changeset	190 unicode->str[0] < 256U &&
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	191 unicode_latin1[unicode->str[0]] == unicode)) {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	192 PyErr_SetString(PyExc_SystemError,
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	193 "can't resize shared unicode objects");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	194 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	195 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	196
38335 f9332ad4287a needforspeed: partition implementation, part two. Fredrik Lundh <fredrik@pythonware.com> parents: 38304 diff changeset	197 /* We allocate one more byte to make sure the string is Ux0000 terminated.
f9332ad4287a needforspeed: partition implementation, part two. Fredrik Lundh <fredrik@pythonware.com> parents: 38304 diff changeset	198 The overallocation is also used by fastsearch, which assumes that it's
38405 3128cfada6b4 Comment typo Andrew M. Kuchling <amk@amk.ca> parents: 38404 diff changeset	199 safe to look at str[length] (without making any assumptions about what
38335 f9332ad4287a needforspeed: partition implementation, part two. Fredrik Lundh <fredrik@pythonware.com> parents: 38304 diff changeset	200 it contains). */
f9332ad4287a needforspeed: partition implementation, part two. Fredrik Lundh <fredrik@pythonware.com> parents: 38304 diff changeset	201
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	202 oldstr = unicode->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	203 PyMem_RESIZE(unicode->str, Py_UNICODE, length + 1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	204 if (!unicode->str) {
37596 1b078493d538 More low-hanging fruit. Still need to re-arrange some code (or find a better Anthony Baxter <anthonybaxter@gmail.com> parents: 37306 diff changeset	205 unicode->str = (Py_UNICODE *)oldstr;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	206 PyErr_NoMemory();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	207 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	208 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	209 unicode->str[length] = 0;
37662 0678ce9fb6b2 Change more ints to Py_ssize_t. Martin v. Löwis <martin@v.loewis.de> parents: 37661 diff changeset	210 unicode->length = length;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	211
12329 3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	212 reset:
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	213 /* Reset the object caches */
13837 2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	214 if (unicode->defenc) {
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	215 Py_DECREF(unicode->defenc);
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	216 unicode->defenc = NULL;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	217 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	218 unicode->hash = -1;
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	219
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	220 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	221 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	222
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	223 /* We allocate one more byte to make sure the string is
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	224 Ux0000 terminated -- XXX is this needed ?
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	225
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	226 XXX This allocator could further be enhanced by assuring that the
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	227 free list never reduces its size below 1.
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	228
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	229 */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	230
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	231 static
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	232 PyUnicodeObject *_PyUnicode_New(Py_ssize_t length)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	233 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	234 register PyUnicodeObject *unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	235
38426 9f3375e33db9 fixed typo Andrew Dalke <dalke@dalkescientific.com> parents: 38424 diff changeset	236 /* Optimization for empty strings */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	237 if (length == 0 && unicode_empty != NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	238 Py_INCREF(unicode_empty);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	239 return unicode_empty;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	240 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	241
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	242 /* Unicode freelist & memory allocation */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	243 if (unicode_freelist) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	244 unicode = unicode_freelist;
12939 88004f25dc25 Vladimir MARANGOZOV <Vladimir.Marangozov@inrialpes.fr>: Marc-André Lemburg <mal@egenix.com> parents: 12924 diff changeset	245 unicode_freelist = (PyUnicodeObject *)unicode;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	246 unicode_freelist_size--;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	247 if (unicode->str) {
12329 3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	248 /* Keep-Alive optimization: we only upsize the buffer,
3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	249 never downsize it. */
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	250 if ((unicode->length < length) &&
30431 cd0c3f916b34 Change checks of PyUnicode_Resize() return value for clarity. Jeremy Hylton <jeremy@alum.mit.edu> parents: 30216 diff changeset	251 unicode_resize(unicode, length) < 0) {
12576 1c7c2dd1beb1 Vladimir Marangozov's long-awaited malloc restructuring. Guido van Rossum <guido@python.org> parents: 12568 diff changeset	252 PyMem_DEL(unicode->str);
12511 8747f1c8b5fb Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12369 diff changeset	253 goto onError;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	254 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	255 }
18120 c29f3135a20c Fix a mis-indentation in _PyUnicode_New() that caused me to stare at Guido van Rossum <guido@python.org> parents: 18051 diff changeset	256 else {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	257 unicode->str = PyMem_NEW(Py_UNICODE, length + 1);
18120 c29f3135a20c Fix a mis-indentation in _PyUnicode_New() that caused me to stare at Guido van Rossum <guido@python.org> parents: 18051 diff changeset	258 }
c29f3135a20c Fix a mis-indentation in _PyUnicode_New() that caused me to stare at Guido van Rossum <guido@python.org> parents: 18051 diff changeset	259 PyObject_INIT(unicode, &PyUnicode_Type);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	260 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	261 else {
22981 dc1fb19d990d Remove PyMalloc_. Neil Schemenauer <nascheme@enme.ucalgary.ca>* parents: 22938 diff changeset	262 unicode = PyObject_New(PyUnicodeObject, &PyUnicode_Type);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	263 if (unicode == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	264 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	265 unicode->str = PyMem_NEW(Py_UNICODE, length + 1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	266 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	267
12511 8747f1c8b5fb Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12369 diff changeset	268 if (!unicode->str) {
8747f1c8b5fb Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12369 diff changeset	269 PyErr_NoMemory();
12025 7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said: Barry Warsaw <barry@python.org> parents: 11997 diff changeset	270 goto onError;
12511 8747f1c8b5fb Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12369 diff changeset	271 }
30439 30e495efe90a Double-fix of crash in Unicode freelist handling. Jeremy Hylton <jeremy@alum.mit.edu> parents: 30431 diff changeset	272 /* Initialize the first element to guard against cases where
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	273 * the caller fails before initializing str -- unicode_resize()
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	274 * reads str[0], and the Keep-Alive optimization can keep memory
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	275 * allocated for str alive across a call to unicode_dealloc(unicode).
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	276 * We don't want unicode_resize to read uninitialized memory in
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	277 * that case.
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	278 */
30439 30e495efe90a Double-fix of crash in Unicode freelist handling. Jeremy Hylton <jeremy@alum.mit.edu> parents: 30431 diff changeset	279 unicode->str[0] = 0;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	280 unicode->str[length] = 0;
37668 4342e441b11c Remove another INT_MAX limitation Martin v. Löwis <martin@v.loewis.de> parents: 37662 diff changeset	281 unicode->length = length;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	282 unicode->hash = -1;
13837 2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	283 unicode->defenc = NULL;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	284 return unicode;
12025 7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said: Barry Warsaw <barry@python.org> parents: 11997 diff changeset	285
7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said: Barry Warsaw <barry@python.org> parents: 11997 diff changeset	286 onError:
7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said: Barry Warsaw <barry@python.org> parents: 11997 diff changeset	287 _Py_ForgetReference((PyObject *)unicode);
22981 dc1fb19d990d Remove PyMalloc_. Neil Schemenauer <nascheme@enme.ucalgary.ca>* parents: 22938 diff changeset	288 PyObject_Del(unicode);
12025 7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said: Barry Warsaw <barry@python.org> parents: 11997 diff changeset	289 return NULL;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	290 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	291
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	292 static
20277 2983b54256c7 Enable GC for new-style instances. This touches lots of files, since Guido van Rossum <guido@python.org> parents: 20018 diff changeset	293 void unicode_dealloc(register PyUnicodeObject *unicode)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	294 {
21262 e8fea95a82a4 Fix for #489669 (Neil Norwitz): memory leak in test_descr (unicode). Guido van Rossum <guido@python.org> parents: 21082 diff changeset	295 if (PyUnicode_CheckExact(unicode) &&
e8fea95a82a4 Fix for #489669 (Neil Norwitz): memory leak in test_descr (unicode). Guido van Rossum <guido@python.org> parents: 21082 diff changeset	296 unicode_freelist_size < MAX_UNICODE_FREELIST_SIZE) {
12329 3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	297 /* Keep-Alive optimization */
3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	298 if (unicode->length >= KEEPALIVE_SIZE_LIMIT) {
12576 1c7c2dd1beb1 Vladimir Marangozov's long-awaited malloc restructuring. Guido van Rossum <guido@python.org> parents: 12568 diff changeset	299 PyMem_DEL(unicode->str);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	300 unicode->str = NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	301 unicode->length = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	302 }
13837 2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	303 if (unicode->defenc) {
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	304 Py_DECREF(unicode->defenc);
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	305 unicode->defenc = NULL;
12329 3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	306 }
3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	307 /* Add to free list */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	308 (PyUnicodeObject *)unicode = unicode_freelist;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	309 unicode_freelist = unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	310 unicode_freelist_size++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	311 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	312 else {
12576 1c7c2dd1beb1 Vladimir Marangozov's long-awaited malloc restructuring. Guido van Rossum <guido@python.org> parents: 12568 diff changeset	313 PyMem_DEL(unicode->str);
13837 2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	314 Py_XDECREF(unicode->defenc);
21262 e8fea95a82a4 Fix for #489669 (Neil Norwitz): memory leak in test_descr (unicode). Guido van Rossum <guido@python.org> parents: 21082 diff changeset	315 unicode->ob_type->tp_free((PyObject *)unicode);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	316 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	317 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	318
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	319 int PyUnicode_Resize(PyObject **unicode, Py_ssize_t length)
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	320 {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	321 register PyUnicodeObject *v;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	322
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	323 /* Argument checks */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	324 if (unicode == NULL) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	325 PyErr_BadInternalCall();
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	326 return -1;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	327 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	328 v = (PyUnicodeObject )unicode;
25719 f532e9b69551 Fix a nasty endcase reported by Armin Rigo in SF bug 618623: Guido van Rossum <guido@python.org> parents: 25375 diff changeset	329 if (v == NULL \|\| !PyUnicode_Check(v) \|\| v->ob_refcnt != 1 \|\| length < 0) {
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	330 PyErr_BadInternalCall();
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	331 return -1;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	332 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	333
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	334 /* Resizing unicode_empty and single character objects is not
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	335 possible since these are being shared. We simply return a fresh
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	336 copy with the same Unicode content. */
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	337 if (v->length != length &&
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	338 (v == unicode_empty \|\| v->length == 1)) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	339 PyUnicodeObject *w = _PyUnicode_New(length);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	340 if (w == NULL)
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	341 return -1;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	342 Py_UNICODE_COPY(w->str, v->str,
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	343 length < v->length ? length : v->length);
28242 fe1d571c3f80 Sf patch #700047: unicode object leaks refcount on resizing Raymond Hettinger <python@rcn.com> parents: 27826 diff changeset	344 Py_DECREF(*unicode);
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	345 unicode = (PyObject )w;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	346 return 0;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	347 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	348
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	349 /* Note that we don't have to modify *unicode for unshared Unicode
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	350 objects, since we can modify them in-place. */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	351 return unicode_resize(v, length);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	352 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	353
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	354 /* Internal API for use in unicodeobject.c only ! */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	355 #define _PyUnicode_Resize(unicodevar, length) \
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	356 PyUnicode_Resize(((PyObject **)(unicodevar)), length)
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	357
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	358 PyObject PyUnicode_FromUnicode(const Py_UNICODE u,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	359 Py_ssize_t size)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	360 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	361 PyUnicodeObject *unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	362
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	363 /* If the Unicode data is known at construction time, we can apply
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	364 some optimizations which share commonly used objects. */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	365 if (u != NULL) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	366
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	367 /* Optimization for empty strings */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	368 if (size == 0 && unicode_empty != NULL) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	369 Py_INCREF(unicode_empty);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	370 return (PyObject *)unicode_empty;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	371 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	372
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	373 /* Single character Unicode objects in the Latin-1 range are
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	374 shared when using this constructor */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	375 if (size == 1 && *u < 256) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	376 unicode = unicode_latin1[*u];
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	377 if (!unicode) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	378 unicode = _PyUnicode_New(1);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	379 if (!unicode)
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	380 return NULL;
18051 e897550b4748 Fixes [ #430986 ] Buglet in PyUnicode_FromUnicode. Marc-André Lemburg <mal@egenix.com> parents: 17995 diff changeset	381 unicode->str[0] = *u;
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	382 unicode_latin1[*u] = unicode;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	383 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	384 Py_INCREF(unicode);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	385 return (PyObject *)unicode;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	386 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	387 }
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	388
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	389 unicode = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	390 if (!unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	391 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	392
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	393 /* Copy the Unicode data into the new object */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	394 if (u != NULL)
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	395 Py_UNICODE_COPY(unicode->str, u, size);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	396
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	397 return (PyObject *)unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	398 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	399
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	400 #ifdef HAVE_WCHAR_H
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	401
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	402 PyObject PyUnicode_FromWideChar(register const wchar_t w,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	403 Py_ssize_t size)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	404 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	405 PyUnicodeObject *unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	406
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	407 if (w == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	408 PyErr_BadInternalCall();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	409 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	410 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	411
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	412 unicode = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	413 if (!unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	414 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	415
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	416 /* Copy the wchar_t data into the new object */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	417 #ifdef HAVE_USABLE_WCHAR_T
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	418 memcpy(unicode->str, w, size * sizeof(wchar_t));
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	419 #else
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	420 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	421 register Py_UNICODE *u;
37662 0678ce9fb6b2 Change more ints to Py_ssize_t. Martin v. Löwis <martin@v.loewis.de> parents: 37661 diff changeset	422 register Py_ssize_t i;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	423 u = PyUnicode_AS_UNICODE(unicode);
33766 1907ad54d351 Applied patch for [ 1047269 ] Buffer overwrite in PyUnicode_AsWideChar. Marc-André Lemburg <mal@egenix.com> parents: 33499 diff changeset	424 for (i = size; i > 0; i--)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	425 u++ = w++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	426 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	427 #endif
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	428
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	429 return (PyObject *)unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	430 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	431
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	432 Py_ssize_t PyUnicode_AsWideChar(PyUnicodeObject *unicode,
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	433 wchar_t *w,
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	434 Py_ssize_t size)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	435 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	436 if (unicode == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	437 PyErr_BadInternalCall();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	438 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	439 }
34016 264e45baacd5 Correct the handling of 0-termination of PyUnicode_AsWideChar() Marc-André Lemburg <mal@egenix.com> parents: 33766 diff changeset	440
264e45baacd5 Correct the handling of 0-termination of PyUnicode_AsWideChar() Marc-André Lemburg <mal@egenix.com> parents: 33766 diff changeset	441 /* If possible, try to copy the 0-termination as well */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	442 if (size > PyUnicode_GET_SIZE(unicode))
34016 264e45baacd5 Correct the handling of 0-termination of PyUnicode_AsWideChar() Marc-André Lemburg <mal@egenix.com> parents: 33766 diff changeset	443 size = PyUnicode_GET_SIZE(unicode) + 1;
264e45baacd5 Correct the handling of 0-termination of PyUnicode_AsWideChar() Marc-André Lemburg <mal@egenix.com> parents: 33766 diff changeset	444
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	445 #ifdef HAVE_USABLE_WCHAR_T
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	446 memcpy(w, unicode->str, size * sizeof(wchar_t));
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	447 #else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	448 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	449 register Py_UNICODE *u;
37662 0678ce9fb6b2 Change more ints to Py_ssize_t. Martin v. Löwis <martin@v.loewis.de> parents: 37661 diff changeset	450 register Py_ssize_t i;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	451 u = PyUnicode_AS_UNICODE(unicode);
33766 1907ad54d351 Applied patch for [ 1047269 ] Buffer overwrite in PyUnicode_AsWideChar. Marc-André Lemburg <mal@egenix.com> parents: 33499 diff changeset	452 for (i = size; i > 0; i--)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	453 w++ = u++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	454 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	455 #endif
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	456
34016 264e45baacd5 Correct the handling of 0-termination of PyUnicode_AsWideChar() Marc-André Lemburg <mal@egenix.com> parents: 33766 diff changeset	457 if (size > PyUnicode_GET_SIZE(unicode))
264e45baacd5 Correct the handling of 0-termination of PyUnicode_AsWideChar() Marc-André Lemburg <mal@egenix.com> parents: 33766 diff changeset	458 return PyUnicode_GET_SIZE(unicode);
264e45baacd5 Correct the handling of 0-termination of PyUnicode_AsWideChar() Marc-André Lemburg <mal@egenix.com> parents: 33766 diff changeset	459 else
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	460 return size;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	461 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	462
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	463 #endif
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	464
24737 eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	465 PyObject *PyUnicode_FromOrdinal(int ordinal)
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	466 {
31873 227cb7402c6e SF Patch #926375: Remove a useless UTF-16 support code that is never Hye-Shik Chang <hyeshik@gmail.com> parents: 31455 diff changeset	467 Py_UNICODE s[1];
24737 eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	468
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	469 #ifdef Py_UNICODE_WIDE
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	470 if (ordinal < 0 \|\| ordinal > 0x10ffff) {
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	471 PyErr_SetString(PyExc_ValueError,
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	472 "unichr() arg not in range(0x110000) "
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	473 "(wide Python build)");
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	474 return NULL;
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	475 }
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	476 #else
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	477 if (ordinal < 0 \|\| ordinal > 0xffff) {
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	478 PyErr_SetString(PyExc_ValueError,
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	479 "unichr() arg not in range(0x10000) "
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	480 "(narrow Python build)");
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	481 return NULL;
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	482 }
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	483 #endif
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	484
31873 227cb7402c6e SF Patch #926375: Remove a useless UTF-16 support code that is never Hye-Shik Chang <hyeshik@gmail.com> parents: 31455 diff changeset	485 s[0] = (Py_UNICODE)ordinal;
227cb7402c6e SF Patch #926375: Remove a useless UTF-16 support code that is never Hye-Shik Chang <hyeshik@gmail.com> parents: 31455 diff changeset	486 return PyUnicode_FromUnicode(s, 1);
24737 eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	487 }
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	488
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	489 PyObject PyUnicode_FromObject(register PyObject obj)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	490 {
20554 0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	491 /* XXX Perhaps we should make this API an alias of
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	492 PyObject_Unicode() instead ?! */
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	493 if (PyUnicode_CheckExact(obj)) {
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	494 Py_INCREF(obj);
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	495 return obj;
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	496 }
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	497 if (PyUnicode_Check(obj)) {
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	498 /* For a Unicode subtype that's not a Unicode object,
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	499 return a true Unicode object with the same data. */
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	500 return PyUnicode_FromUnicode(PyUnicode_AS_UNICODE(obj),
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	501 PyUnicode_GET_SIZE(obj));
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	502 }
13401 8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	503 return PyUnicode_FromEncodedObject(obj, NULL, "strict");
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	504 }
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	505
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	506 PyObject PyUnicode_FromEncodedObject(register PyObject obj,
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	507 const char *encoding,
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	508 const char *errors)
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	509 {
19999 1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	510 const char *s = NULL;
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	511 Py_ssize_t len;
13401 8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	512 PyObject *v;
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	513
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	514 if (obj == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	515 PyErr_BadInternalCall();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	516 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	517 }
13401 8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	518
20554 0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	519 #if 0
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	520 /* For b/w compatibility we also accept Unicode objects provided
20563 9058e49c6c57 Additional test and documentation for the unicode() changes. Marc-André Lemburg <mal@egenix.com> parents: 20554 diff changeset	521 that no encodings is given and then redirect to
9058e49c6c57 Additional test and documentation for the unicode() changes. Marc-André Lemburg <mal@egenix.com> parents: 20554 diff changeset	522 PyObject_Unicode() which then applies the additional logic for
9058e49c6c57 Additional test and documentation for the unicode() changes. Marc-André Lemburg <mal@egenix.com> parents: 20554 diff changeset	523 Unicode subclasses.
20554 0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	524
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	525 NOTE: This API should really only be used for object which
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	526 represent encoded Unicode !
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	527
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	528 */
19999 1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	529 if (PyUnicode_Check(obj)) {
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	530 if (encoding) {
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	531 PyErr_SetString(PyExc_TypeError,
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	532 "decoding Unicode is not supported");
20554 0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	533 return NULL;
19999 1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	534 }
20554 0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	535 return PyObject_Unicode(obj);
19999 1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	536 }
20554 0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	537 #else
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	538 if (PyUnicode_Check(obj)) {
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	539 PyErr_SetString(PyExc_TypeError,
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	540 "decoding Unicode is not supported");
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	541 return NULL;
13401 8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	542 }
20554 0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	543 #endif
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	544
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	545 /* Coerce object */
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	546 if (PyString_Check(obj)) {
19999 1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	547 s = PyString_AS_STRING(obj);
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	548 len = PyString_GET_SIZE(obj);
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	549 }
20554 0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	550 else if (PyObject_AsCharBuffer(obj, &s, &len)) {
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	551 /* Overwrite the error message with something more useful in
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	552 case of a TypeError. */
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	553 if (PyErr_ExceptionMatches(PyExc_TypeError))
19999 1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	554 PyErr_Format(PyExc_TypeError,
20554 0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	555 "coercing to Unicode: need string or buffer, "
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	556 "%.80s found",
19999 1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	557 obj->ob_type->tp_name);
13401 8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	558 goto onError;
12248 70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	559 }
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	560
13401 8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	561 /* Convert to Unicode */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	562 if (len == 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	563 Py_INCREF(unicode_empty);
13401 8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	564 v = (PyObject *)unicode_empty;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	565 }
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	566 else
13401 8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	567 v = PyUnicode_Decode(s, len, encoding, errors);
15952 1f92d63b20f0 This patch adds a new builtin unistr() which behaves like str() Marc-André Lemburg <mal@egenix.com> parents: 15910 diff changeset	568
13401 8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	569 return v;
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	570
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	571 onError:
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	572 return NULL;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	573 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	574
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	575 PyObject PyUnicode_Decode(const char s,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	576 Py_ssize_t size,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	577 const char *encoding,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	578 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	579 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	580 PyObject buffer = NULL, unicode;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	581
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	582 if (encoding == NULL)
12645 c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	583 encoding = PyUnicode_GetDefaultEncoding();
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	584
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	585 /* Shortcuts for common default encodings */
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	586 if (strcmp(encoding, "utf-8") == 0)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	587 return PyUnicode_DecodeUTF8(s, size, errors);
12645 c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	588 else if (strcmp(encoding, "latin-1") == 0)
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	589 return PyUnicode_DecodeLatin1(s, size, errors);
29706 a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without Mark Hammond <mhammond@skippinet.com.au> parents: 29641 diff changeset	590 #if defined(MS_WINDOWS) && defined(HAVE_USABLE_WCHAR_T)
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without Mark Hammond <mhammond@skippinet.com.au> parents: 29641 diff changeset	591 else if (strcmp(encoding, "mbcs") == 0)
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without Mark Hammond <mhammond@skippinet.com.au> parents: 29641 diff changeset	592 return PyUnicode_DecodeMBCS(s, size, errors);
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without Mark Hammond <mhammond@skippinet.com.au> parents: 29641 diff changeset	593 #endif
12645 c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	594 else if (strcmp(encoding, "ascii") == 0)
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	595 return PyUnicode_DecodeASCII(s, size, errors);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	596
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	597 /* Decode via the codec registry */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	598 buffer = PyBuffer_FromMemory((void *)s, size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	599 if (buffer == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	600 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	601 unicode = PyCodec_Decode(buffer, encoding, errors);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	602 if (unicode == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	603 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	604 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	605 PyErr_Format(PyExc_TypeError,
12320 5de0b939fe61 Skip Montanaro: add string precisions to calls to PyErr_Format Guido van Rossum <guido@python.org> parents: 12275 diff changeset	606 "decoder did not return an unicode object (type=%.400s)",
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	607 unicode->ob_type->tp_name);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	608 Py_DECREF(unicode);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	609 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	610 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	611 Py_DECREF(buffer);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	612 return unicode;
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	613
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	614 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	615 Py_XDECREF(buffer);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	616 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	617 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	618
32551 f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	619 PyObject PyUnicode_AsDecodedObject(PyObject unicode,
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	620 const char *encoding,
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	621 const char *errors)
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	622 {
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	623 PyObject *v;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	624
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	625 if (!PyUnicode_Check(unicode)) {
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	626 PyErr_BadArgument();
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	627 goto onError;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	628 }
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	629
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	630 if (encoding == NULL)
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	631 encoding = PyUnicode_GetDefaultEncoding();
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	632
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	633 /* Decode via the codec registry */
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	634 v = PyCodec_Decode(unicode, encoding, errors);
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	635 if (v == NULL)
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	636 goto onError;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	637 return v;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	638
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	639 onError:
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	640 return NULL;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	641 }
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	642
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	643 PyObject PyUnicode_Encode(const Py_UNICODE s,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	644 Py_ssize_t size,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	645 const char *encoding,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	646 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	647 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	648 PyObject v, unicode;
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	649
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	650 unicode = PyUnicode_FromUnicode(s, size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	651 if (unicode == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	652 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	653 v = PyUnicode_AsEncodedString(unicode, encoding, errors);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	654 Py_DECREF(unicode);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	655 return v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	656 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	657
32551 f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	658 PyObject PyUnicode_AsEncodedObject(PyObject unicode,
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	659 const char *encoding,
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	660 const char *errors)
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	661 {
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	662 PyObject *v;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	663
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	664 if (!PyUnicode_Check(unicode)) {
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	665 PyErr_BadArgument();
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	666 goto onError;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	667 }
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	668
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	669 if (encoding == NULL)
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	670 encoding = PyUnicode_GetDefaultEncoding();
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	671
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	672 /* Encode via the codec registry */
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	673 v = PyCodec_Encode(unicode, encoding, errors);
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	674 if (v == NULL)
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	675 goto onError;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	676 return v;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	677
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	678 onError:
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	679 return NULL;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	680 }
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	681
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	682 PyObject PyUnicode_AsEncodedString(PyObject unicode,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	683 const char *encoding,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	684 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	685 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	686 PyObject *v;
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	687
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	688 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	689 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	690 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	691 }
12645 c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	692
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	693 if (encoding == NULL)
12645 c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	694 encoding = PyUnicode_GetDefaultEncoding();
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	695
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	696 /* Shortcuts for common default encodings */
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	697 if (errors == NULL) {
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	698 if (strcmp(encoding, "utf-8") == 0)
17995 837825a779c1 fix bogus indentation Jeremy Hylton <jeremy@alum.mit.edu> parents: 17887 diff changeset	699 return PyUnicode_AsUTF8String(unicode);
12645 c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	700 else if (strcmp(encoding, "latin-1") == 0)
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	701 return PyUnicode_AsLatin1String(unicode);
29706 a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without Mark Hammond <mhammond@skippinet.com.au> parents: 29641 diff changeset	702 #if defined(MS_WINDOWS) && defined(HAVE_USABLE_WCHAR_T)
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without Mark Hammond <mhammond@skippinet.com.au> parents: 29641 diff changeset	703 else if (strcmp(encoding, "mbcs") == 0)
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without Mark Hammond <mhammond@skippinet.com.au> parents: 29641 diff changeset	704 return PyUnicode_AsMBCSString(unicode);
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without Mark Hammond <mhammond@skippinet.com.au> parents: 29641 diff changeset	705 #endif
12645 c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	706 else if (strcmp(encoding, "ascii") == 0)
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	707 return PyUnicode_AsASCIIString(unicode);
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	708 }
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	709
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	710 /* Encode via the codec registry */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	711 v = PyCodec_Encode(unicode, encoding, errors);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	712 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	713 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	714 if (!PyString_Check(v)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	715 PyErr_Format(PyExc_TypeError,
12320 5de0b939fe61 Skip Montanaro: add string precisions to calls to PyErr_Format Guido van Rossum <guido@python.org> parents: 12275 diff changeset	716 "encoder did not return a string object (type=%.400s)",
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	717 v->ob_type->tp_name);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	718 Py_DECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	719 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	720 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	721 return v;
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	722
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	723 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	724 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	725 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	726
13837 2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	727 PyObject _PyUnicode_AsDefaultEncodedString(PyObject unicode,
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	728 const char *errors)
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	729 {
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	730 PyObject v = ((PyUnicodeObject )unicode)->defenc;
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	731
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	732 if (v)
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	733 return v;
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	734 v = PyUnicode_AsEncodedString(unicode, NULL, errors);
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	735 if (v && errors == NULL)
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	736 ((PyUnicodeObject *)unicode)->defenc = v;
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	737 return v;
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	738 }
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	739
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	740 Py_UNICODE PyUnicode_AsUnicode(PyObject unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	741 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	742 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	743 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	744 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	745 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	746 return PyUnicode_AS_UNICODE(unicode);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	747
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	748 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	749 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	750 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	751
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	752 Py_ssize_t PyUnicode_GetSize(PyObject *unicode)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	753 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	754 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	755 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	756 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	757 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	758 return PyUnicode_GET_SIZE(unicode);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	759
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	760 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	761 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	762 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	763
13660 a1ce07218781 Miscelaneous ANSIfications. I'm assuming here 'main' should take (int, Thomas Wouters <thomas@python.org> parents: 13635 diff changeset	764 const char *PyUnicode_GetDefaultEncoding(void)
12645 c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	765 {
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	766 return unicode_default_encoding;
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	767 }
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	768
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	769 int PyUnicode_SetDefaultEncoding(const char *encoding)
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	770 {
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	771 PyObject *v;
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	772
12645 c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	773 /* Make sure the encoding is valid. As side effect, this also
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	774 loads the encoding into the codec registry cache. */
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	775 v = _PyCodec_Lookup(encoding);
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	776 if (v == NULL)
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	777 goto onError;
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	778 Py_DECREF(v);
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	779 strncpy(unicode_default_encoding,
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	780 encoding,
12645 c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	781 sizeof(unicode_default_encoding));
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	782 return 0;
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	783
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	784 onError:
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	785 return -1;
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	786 }
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	787
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	788 /* error handling callback helper:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	789 build arguments, call the callback and check the arguments,
35806 44fe157255bc fix typos, mostly in comments Fred Drake <fdrake@acm.org> parents: 35758 diff changeset	790 if no exception occurred, copy the replacement to the output
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	791 and adjust various state variables.
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	792 return 0 on success, -1 on error
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	793 */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	794
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	795 static
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	796 int unicode_decode_call_errorhandler(const char errors, PyObject *errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	797 const char encoding, const char reason,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	798 const char input, Py_ssize_t insize, Py_ssize_t startinpos, Py_ssize_t endinpos, PyObject exceptionObject, const char *inptr,
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	799 PyObject *output, Py_ssize_t outpos, Py_UNICODE **outptr)
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	800 {
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	801 static char *argparse = "O!n;decoding error handler must return (unicode, int) tuple";
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	802
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	803 PyObject *restuple = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	804 PyObject *repunicode = NULL;
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	805 Py_ssize_t outsize = PyUnicode_GET_SIZE(*output);
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	806 Py_ssize_t requiredsize;
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	807 Py_ssize_t newpos;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	808 Py_UNICODE *repptr;
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	809 Py_ssize_t repsize;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	810 int res = -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	811
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	812 if (*errorHandler == NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	813 *errorHandler = PyCodec_LookupError(errors);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	814 if (*errorHandler == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	815 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	816 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	817
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	818 if (*exceptionObject == NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	819 *exceptionObject = PyUnicodeDecodeError_Create(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	820 encoding, input, insize, startinpos, endinpos, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	821 if (*exceptionObject == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	822 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	823 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	824 else {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	825 if (PyUnicodeDecodeError_SetStart(exceptionObject, startinpos))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	826 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	827 if (PyUnicodeDecodeError_SetEnd(exceptionObject, endinpos))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	828 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	829 if (PyUnicodeDecodeError_SetReason(*exceptionObject, reason))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	830 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	831 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	832
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	833 restuple = PyObject_CallFunctionObjArgs(errorHandler, exceptionObject, NULL);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	834 if (restuple == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	835 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	836 if (!PyTuple_Check(restuple)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	837 PyErr_Format(PyExc_TypeError, &argparse[4]);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	838 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	839 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	840 if (!PyArg_ParseTuple(restuple, argparse, &PyUnicode_Type, &repunicode, &newpos))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	841 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	842 if (newpos<0)
27555 c267f28724dd Change the treatment of positions returned by PEP293 Walter Dörwald <walter@livinglogic.de> parents: 27521 diff changeset	843 newpos = insize+newpos;
c267f28724dd Change the treatment of positions returned by PEP293 Walter Dörwald <walter@livinglogic.de> parents: 27521 diff changeset	844 if (newpos<0 \|\| newpos>insize) {
36510 0f3894bb2ada Support %zd in PyErr_Format and PyString_FromFormat. Martin v. Löwis <martin@v.loewis.de> parents: 36508 diff changeset	845 PyErr_Format(PyExc_IndexError, "position %zd from error handler out of bounds", newpos);
27555 c267f28724dd Change the treatment of positions returned by PEP293 Walter Dörwald <walter@livinglogic.de> parents: 27521 diff changeset	846 goto onError;
c267f28724dd Change the treatment of positions returned by PEP293 Walter Dörwald <walter@livinglogic.de> parents: 27521 diff changeset	847 }
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	848
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	849 /* need more space? (at least enough for what we
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	850 have+the replacement+the rest of the string (starting
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	851 at the new input position), so we won't have to check space
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	852 when there are no errors in the rest of the string) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	853 repptr = PyUnicode_AS_UNICODE(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	854 repsize = PyUnicode_GET_SIZE(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	855 requiredsize = *outpos + repsize + insize-newpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	856 if (requiredsize > outsize) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	857 if (requiredsize<2*outsize)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	858 requiredsize = 2*outsize;
30431 cd0c3f916b34 Change checks of PyUnicode_Resize() return value for clarity. Jeremy Hylton <jeremy@alum.mit.edu> parents: 30216 diff changeset	859 if (PyUnicode_Resize(output, requiredsize) < 0)
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	860 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	861 outptr = PyUnicode_AS_UNICODE(output) + *outpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	862 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	863 *endinpos = newpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	864 *inptr = input + newpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	865 Py_UNICODE_COPY(*outptr, repptr, repsize);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	866 *outptr += repsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	867 *outpos += repsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	868 /* we made it! */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	869 res = 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	870
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	871 onError:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	872 Py_XDECREF(restuple);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	873 return res;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	874 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	875
19998 8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	876 /* --- UTF-7 Codec -------------------------------------------------------- */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	877
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	878 /* see RFC2152 for details */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	879
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	880 static
19998 8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	881 char utf7_special[128] = {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	882 /* indicate whether a UTF-7 character is special i.e. cannot be directly
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	883 encoded:
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	884 0 - not special
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	885 1 - special
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	886 2 - whitespace (optional)
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	887 3 - RFC2152 Set O (optional) */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	888 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 1, 1, 2, 1, 1,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	889 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	890 2, 3, 3, 3, 3, 3, 3, 0, 0, 0, 3, 1, 0, 0, 0, 1,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	891 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 3, 3, 3, 0,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	892 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	893 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 1, 3, 3, 3,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	894 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	895 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 3, 3, 1, 1,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	896
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	897 };
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	898
35728 9ed43e637ae7 Bug fix for [ 1331062 ] utf 7 codec broken. Marc-André Lemburg <mal@egenix.com> parents: 35702 diff changeset	899 /* Note: The comparison (c) <= 0 is a trick to work-around gcc
9ed43e637ae7 Bug fix for [ 1331062 ] utf 7 codec broken. Marc-André Lemburg <mal@egenix.com> parents: 35702 diff changeset	900 warnings about the comparison always being false; since
9ed43e637ae7 Bug fix for [ 1331062 ] utf 7 codec broken. Marc-André Lemburg <mal@egenix.com> parents: 35702 diff changeset	901 utf7_special[0] is 1, we can safely make that one comparison
9ed43e637ae7 Bug fix for [ 1331062 ] utf 7 codec broken. Marc-André Lemburg <mal@egenix.com> parents: 35702 diff changeset	902 true */
9ed43e637ae7 Bug fix for [ 1331062 ] utf 7 codec broken. Marc-André Lemburg <mal@egenix.com> parents: 35702 diff changeset	903
19998 8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	904 #define SPECIAL(c, encodeO, encodeWS) \
35728 9ed43e637ae7 Bug fix for [ 1331062 ] utf 7 codec broken. Marc-André Lemburg <mal@egenix.com> parents: 35702 diff changeset	905 ((c) > 127 \|\| (c) <= 0 \|\| utf7_special[(c)] == 1 \|\| \
35729 6913abb7047a Whitespace corrections. Marc-André Lemburg <mal@egenix.com> parents: 35728 diff changeset	906 (encodeWS && (utf7_special[(c)] == 2)) \|\| \
19998 8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	907 (encodeO && (utf7_special[(c)] == 3)))
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	908
35728 9ed43e637ae7 Bug fix for [ 1331062 ] utf 7 codec broken. Marc-André Lemburg <mal@egenix.com> parents: 35702 diff changeset	909 #define B64(n) \
9ed43e637ae7 Bug fix for [ 1331062 ] utf 7 codec broken. Marc-André Lemburg <mal@egenix.com> parents: 35702 diff changeset	910 ("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/"[(n) & 0x3f])
9ed43e637ae7 Bug fix for [ 1331062 ] utf 7 codec broken. Marc-André Lemburg <mal@egenix.com> parents: 35702 diff changeset	911 #define B64CHAR(c) \
9ed43e637ae7 Bug fix for [ 1331062 ] utf 7 codec broken. Marc-André Lemburg <mal@egenix.com> parents: 35702 diff changeset	912 (isalnum(c) \|\| (c) == '+' \|\| (c) == '/')
9ed43e637ae7 Bug fix for [ 1331062 ] utf 7 codec broken. Marc-André Lemburg <mal@egenix.com> parents: 35702 diff changeset	913 #define UB64(c) \
9ed43e637ae7 Bug fix for [ 1331062 ] utf 7 codec broken. Marc-André Lemburg <mal@egenix.com> parents: 35702 diff changeset	914 ((c) == '+' ? 62 : (c) == '/' ? 63 : (c) >= 'a' ? \
9ed43e637ae7 Bug fix for [ 1331062 ] utf 7 codec broken. Marc-André Lemburg <mal@egenix.com> parents: 35702 diff changeset	915 (c) - 71 : (c) >= 'A' ? (c) - 65 : (c) + 4 )
19998 8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	916
35729 6913abb7047a Whitespace corrections. Marc-André Lemburg <mal@egenix.com> parents: 35728 diff changeset	917 #define ENCODE(out, ch, bits) \
6913abb7047a Whitespace corrections. Marc-André Lemburg <mal@egenix.com> parents: 35728 diff changeset	918 while (bits >= 6) { \
6913abb7047a Whitespace corrections. Marc-André Lemburg <mal@egenix.com> parents: 35728 diff changeset	919 *out++ = B64(ch >> (bits-6)); \
6913abb7047a Whitespace corrections. Marc-André Lemburg <mal@egenix.com> parents: 35728 diff changeset	920 bits -= 6; \
6913abb7047a Whitespace corrections. Marc-André Lemburg <mal@egenix.com> parents: 35728 diff changeset	921 }
6913abb7047a Whitespace corrections. Marc-André Lemburg <mal@egenix.com> parents: 35728 diff changeset	922
6913abb7047a Whitespace corrections. Marc-André Lemburg <mal@egenix.com> parents: 35728 diff changeset	923 #define DECODE(out, ch, bits, surrogate) \
6913abb7047a Whitespace corrections. Marc-André Lemburg <mal@egenix.com> parents: 35728 diff changeset	924 while (bits >= 16) { \
6913abb7047a Whitespace corrections. Marc-André Lemburg <mal@egenix.com> parents: 35728 diff changeset	925 Py_UNICODE outCh = (Py_UNICODE) ((ch >> (bits-16)) & 0xffff); \
6913abb7047a Whitespace corrections. Marc-André Lemburg <mal@egenix.com> parents: 35728 diff changeset	926 bits -= 16; \
6913abb7047a Whitespace corrections. Marc-André Lemburg <mal@egenix.com> parents: 35728 diff changeset	927 if (surrogate) { \
35728 9ed43e637ae7 Bug fix for [ 1331062 ] utf 7 codec broken. Marc-André Lemburg <mal@egenix.com> parents: 35702 diff changeset	928 /* We have already generated an error for the high surrogate \
9ed43e637ae7 Bug fix for [ 1331062 ] utf 7 codec broken. Marc-André Lemburg <mal@egenix.com> parents: 35702 diff changeset	929 so let's not bother seeing if the low surrogate is correct or not */ \
35729 6913abb7047a Whitespace corrections. Marc-André Lemburg <mal@egenix.com> parents: 35728 diff changeset	930 surrogate = 0; \
6913abb7047a Whitespace corrections. Marc-André Lemburg <mal@egenix.com> parents: 35728 diff changeset	931 } else if (0xDC00 <= outCh && outCh <= 0xDFFF) { \
19998 8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	932 /* This is a surrogate pair. Unfortunately we can't represent \
35729 6913abb7047a Whitespace corrections. Marc-André Lemburg <mal@egenix.com> parents: 35728 diff changeset	933 it in a 16-bit character */ \
6913abb7047a Whitespace corrections. Marc-André Lemburg <mal@egenix.com> parents: 35728 diff changeset	934 surrogate = 1; \
6913abb7047a Whitespace corrections. Marc-André Lemburg <mal@egenix.com> parents: 35728 diff changeset	935 errmsg = "code pairs are not supported"; \
6913abb7047a Whitespace corrections. Marc-André Lemburg <mal@egenix.com> parents: 35728 diff changeset	936 goto utf7Error; \
6913abb7047a Whitespace corrections. Marc-André Lemburg <mal@egenix.com> parents: 35728 diff changeset	937 } else { \
6913abb7047a Whitespace corrections. Marc-André Lemburg <mal@egenix.com> parents: 35728 diff changeset	938 *out++ = outCh; \
6913abb7047a Whitespace corrections. Marc-André Lemburg <mal@egenix.com> parents: 35728 diff changeset	939 } \
35728 9ed43e637ae7 Bug fix for [ 1331062 ] utf 7 codec broken. Marc-André Lemburg <mal@egenix.com> parents: 35702 diff changeset	940 }
19998 8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	941
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	942 PyObject PyUnicode_DecodeUTF7(const char s,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	943 Py_ssize_t size,
19998 8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	944 const char *errors)
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	945 {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	946 const char *starts = s;
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	947 Py_ssize_t startinpos;
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	948 Py_ssize_t endinpos;
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	949 Py_ssize_t outpos;
19998 8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	950 const char *e;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	951 PyUnicodeObject *unicode;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	952 Py_UNICODE *p;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	953 const char *errmsg = "";
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	954 int inShift = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	955 unsigned int bitsleft = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	956 unsigned long charsleft = 0;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	957 int surrogate = 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	958 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	959 PyObject *exc = NULL;
19998 8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	960
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	961 unicode = _PyUnicode_New(size);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	962 if (!unicode)
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	963 return NULL;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	964 if (size == 0)
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	965 return (PyObject *)unicode;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	966
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	967 p = unicode->str;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	968 e = s + size;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	969
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	970 while (s < e) {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	971 Py_UNICODE ch;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	972 restart:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	973 ch = *s;
19998 8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	974
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	975 if (inShift) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	976 if ((ch == '-') \|\| !B64CHAR(ch)) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	977 inShift = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	978 s++;
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	979
19998 8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	980 /* p, charsleft, bitsleft, surrogate = */ DECODE(p, charsleft, bitsleft, surrogate);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	981 if (bitsleft >= 6) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	982 /* The shift sequence has a partial character in it. If
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	983 bitsleft < 6 then we could just classify it as padding
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	984 but that is not the case here */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	985
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	986 errmsg = "partial character in shift sequence";
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	987 goto utf7Error;
19998 8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	988 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	989 /* According to RFC2152 the remaining bits should be zero. We
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	990 choose to signal an error/insert a replacement character
19998 8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	991 here so indicate the potential of a misencoded character. */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	992
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	993 /* On x86, a << b == a << (b%32) so make sure that bitsleft != 0 */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	994 if (bitsleft && charsleft << (sizeof(charsleft) * 8 - bitsleft)) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	995 errmsg = "non-zero padding bits in shift sequence";
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	996 goto utf7Error;
19998 8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	997 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	999 if (ch == '-') {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1000 if ((s < e) && (*(s) == '-')) {
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	1001 *p++ = '-';
19998 8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1002 inShift = 1;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1003 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1004 } else if (SPECIAL(ch,0,0)) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1005 errmsg = "unexpected special character";
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	1006 goto utf7Error;
19998 8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1007 } else {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1008 *p++ = ch;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1009 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1010 } else {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1011 charsleft = (charsleft << 6) \| UB64(ch);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1012 bitsleft += 6;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1013 s++;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1014 /* p, charsleft, bitsleft, surrogate = */ DECODE(p, charsleft, bitsleft, surrogate);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1015 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1016 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1017 else if ( ch == '+' ) {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1018 startinpos = s-starts;
19998 8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1019 s++;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1020 if (s < e && *s == '-') {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1021 s++;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1022 *p++ = '+';
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1023 } else
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1024 {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1025 inShift = 1;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1026 bitsleft = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1027 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1028 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1029 else if (SPECIAL(ch,0,0)) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1030 errmsg = "unexpected special character";
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1031 s++;
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	1032 goto utf7Error;
19998 8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1033 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1034 else {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1035 *p++ = ch;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1036 s++;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1037 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1038 continue;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1039 utf7Error:
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1040 outpos = p-PyUnicode_AS_UNICODE(unicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1041 endinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1042 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1043 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1044 "utf7", errmsg,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1045 starts, size, &startinpos, &endinpos, &exc, &s,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1046 (PyObject **)&unicode, &outpos, &p))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1047 goto onError;
19998 8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1048 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1049
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1050 if (inShift) {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1051 outpos = p-PyUnicode_AS_UNICODE(unicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1052 endinpos = size;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1053 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1054 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1055 "utf7", "unterminated shift sequence",
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1056 starts, size, &startinpos, &endinpos, &exc, &s,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1057 (PyObject **)&unicode, &outpos, &p))
19998 8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1058 goto onError;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1059 if (s < e)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1060 goto restart;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1061 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1062
30431 cd0c3f916b34 Change checks of PyUnicode_Resize() return value for clarity. Jeremy Hylton <jeremy@alum.mit.edu> parents: 30216 diff changeset	1063 if (_PyUnicode_Resize(&unicode, p - PyUnicode_AS_UNICODE(unicode)) < 0)
19998 8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1064 goto onError;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1065
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1066 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1067 Py_XDECREF(exc);
19998 8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1068 return (PyObject *)unicode;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1069
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1070 onError:
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1071 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1072 Py_XDECREF(exc);
19998 8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1073 Py_DECREF(unicode);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1074 return NULL;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1075 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1076
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1077
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1078 PyObject PyUnicode_EncodeUTF7(const Py_UNICODE s,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	1079 Py_ssize_t size,
19998 8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1080 int encodeSetO,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1081 int encodeWhiteSpace,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1082 const char *errors)
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1083 {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1084 PyObject *v;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1085 /* It might be possible to tighten this worst case */
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	1086 Py_ssize_t cbAllocated = 5 * size;
19998 8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1087 int inShift = 0;
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	1088 Py_ssize_t i = 0;
19998 8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1089 unsigned int bitsleft = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1090 unsigned long charsleft = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1091 char * out;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1092 char * start;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1093
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1094 if (size == 0)
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1095 return PyString_FromStringAndSize(NULL, 0);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1096
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1097 v = PyString_FromStringAndSize(NULL, cbAllocated);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1098 if (v == NULL)
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1099 return NULL;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1100
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1101 start = out = PyString_AS_STRING(v);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1102 for (;i < size; ++i) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1103 Py_UNICODE ch = s[i];
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1104
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1105 if (!inShift) {
31325 b4532c246a3a Cosmetic fix for wrongly indented tabs with ts=4. Hye-Shik Chang <hyeshik@gmail.com> parents: 31252 diff changeset	1106 if (ch == '+') {
b4532c246a3a Cosmetic fix for wrongly indented tabs with ts=4. Hye-Shik Chang <hyeshik@gmail.com> parents: 31252 diff changeset	1107 *out++ = '+';
19998 8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1108 *out++ = '-';
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1109 } else if (SPECIAL(ch, encodeSetO, encodeWhiteSpace)) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1110 charsleft = ch;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1111 bitsleft = 16;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1112 *out++ = '+';
31325 b4532c246a3a Cosmetic fix for wrongly indented tabs with ts=4. Hye-Shik Chang <hyeshik@gmail.com> parents: 31252 diff changeset	1113 /* out, charsleft, bitsleft = */ ENCODE(out, charsleft, bitsleft);
19998 8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1114 inShift = bitsleft > 0;
31325 b4532c246a3a Cosmetic fix for wrongly indented tabs with ts=4. Hye-Shik Chang <hyeshik@gmail.com> parents: 31252 diff changeset	1115 } else {
b4532c246a3a Cosmetic fix for wrongly indented tabs with ts=4. Hye-Shik Chang <hyeshik@gmail.com> parents: 31252 diff changeset	1116 *out++ = (char) ch;
b4532c246a3a Cosmetic fix for wrongly indented tabs with ts=4. Hye-Shik Chang <hyeshik@gmail.com> parents: 31252 diff changeset	1117 }
b4532c246a3a Cosmetic fix for wrongly indented tabs with ts=4. Hye-Shik Chang <hyeshik@gmail.com> parents: 31252 diff changeset	1118 } else {
19998 8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1119 if (!SPECIAL(ch, encodeSetO, encodeWhiteSpace)) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1120 *out++ = B64(charsleft << (6-bitsleft));
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1121 charsleft = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1122 bitsleft = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1123 /* Characters not in the BASE64 set implicitly unshift the sequence
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1124 so no '-' is required, except if the character is itself a '-' */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1125 if (B64CHAR(ch) \|\| ch == '-') {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1126 *out++ = '-';
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1127 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1128 inShift = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1129 *out++ = (char) ch;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1130 } else {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1131 bitsleft += 16;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1132 charsleft = (charsleft << 16) \| ch;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1133 /* out, charsleft, bitsleft = */ ENCODE(out, charsleft, bitsleft);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1134
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1135 /* If the next character is special then we dont' need to terminate
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	1136 the shift sequence. If the next character is not a BASE64 character
19998 8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1137 or '-' then the shift sequence will be terminated implicitly and we
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1138 don't have to insert a '-'. */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1139
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1140 if (bitsleft == 0) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1141 if (i + 1 < size) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1142 Py_UNICODE ch2 = s[i+1];
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1143
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1144 if (SPECIAL(ch2, encodeSetO, encodeWhiteSpace)) {
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	1145
19998 8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1146 } else if (B64CHAR(ch2) \|\| ch2 == '-') {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1147 *out++ = '-';
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1148 inShift = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1149 } else {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1150 inShift = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1151 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1152
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1153 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1154 else {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1155 *out++ = '-';
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1156 inShift = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1157 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1158 }
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	1159 }
19998 8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1160 }
31325 b4532c246a3a Cosmetic fix for wrongly indented tabs with ts=4. Hye-Shik Chang <hyeshik@gmail.com> parents: 31252 diff changeset	1161 }
19998 8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1162 if (bitsleft) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1163 *out++= B64(charsleft << (6-bitsleft) );
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1164 *out++ = '-';
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1165 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1166
23295 6fa76b4e59fe Repair widespread misuse of _PyString_Resize. Since it's clear people Tim Peters <tim.peters@gmail.com> parents: 23294 diff changeset	1167 _PyString_Resize(&v, out - start);
19998 8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1168 return v;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1169 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1170
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1171 #undef SPECIAL
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1172 #undef B64
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1173 #undef B64CHAR
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1174 #undef UB64
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1175 #undef ENCODE
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1176 #undef DECODE
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1177
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1178 /* --- UTF-8 Codec -------------------------------------------------------- */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1179
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	1180 static
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1181 char utf8_code_length[256] = {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1182 /* Map UTF-8 encoded prefix byte to sequence length. zero means
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1183 illegal prefix. see RFC 2279 for details */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1184 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1185 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1186 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1187 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1188 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1189 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1190 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1191 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1192 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1193 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1194 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1195 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1196 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1197 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1198 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1199 4, 4, 4, 4, 4, 4, 4, 4, 5, 5, 5, 5, 6, 6, 0, 0
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1200 };
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1201
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1202 PyObject PyUnicode_DecodeUTF8(const char s,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	1203 Py_ssize_t size,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1204 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1205 {
33428 57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 33311 diff changeset	1206 return PyUnicode_DecodeUTF8Stateful(s, size, errors, NULL);
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 33311 diff changeset	1207 }
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 33311 diff changeset	1208
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 33311 diff changeset	1209 PyObject PyUnicode_DecodeUTF8Stateful(const char s,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	1210 Py_ssize_t size,
33428 57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 33311 diff changeset	1211 const char *errors,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	1212 Py_ssize_t *consumed)
33428 57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 33311 diff changeset	1213 {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1214 const char *starts = s;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1215 int n;
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	1216 Py_ssize_t startinpos;
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	1217 Py_ssize_t endinpos;
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	1218 Py_ssize_t outpos;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1219 const char *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1220 PyUnicodeObject *unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1221 Py_UNICODE *p;
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1222 const char *errmsg = "";
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1223 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1224 PyObject *exc = NULL;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1225
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1226 /* Note: size will always be longer than the resulting Unicode
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1227 character count */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1228 unicode = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1229 if (!unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1230 return NULL;
33428 57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 33311 diff changeset	1231 if (size == 0) {
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 33311 diff changeset	1232 if (consumed)
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 33311 diff changeset	1233 *consumed = 0;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1234 return (PyObject *)unicode;
33428 57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 33311 diff changeset	1235 }
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1236
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1237 /* Unpack UTF-8 encoded data */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1238 p = unicode->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1239 e = s + size;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1240
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1241 while (s < e) {
13411 9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1242 Py_UCS4 ch = (unsigned char)*s;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1243
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1244 if (ch < 0x80) {
13411 9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1245 *p++ = (Py_UNICODE)ch;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1246 s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1247 continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1248 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1249
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1250 n = utf8_code_length[ch];
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1251
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1252 if (s + n > e) {
33428 57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 33311 diff changeset	1253 if (consumed)
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 33311 diff changeset	1254 break;
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 33311 diff changeset	1255 else {
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 33311 diff changeset	1256 errmsg = "unexpected end of data";
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 33311 diff changeset	1257 startinpos = s-starts;
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 33311 diff changeset	1258 endinpos = size;
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 33311 diff changeset	1259 goto utf8Error;
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 33311 diff changeset	1260 }
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1261 }
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1262
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1263 switch (n) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1264
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1265 case 0:
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1266 errmsg = "unexpected code byte";
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1267 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1268 endinpos = startinpos+1;
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1269 goto utf8Error;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1270
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1271 case 1:
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1272 errmsg = "internal error";
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1273 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1274 endinpos = startinpos+1;
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1275 goto utf8Error;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1276
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1277 case 2:
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1278 if ((s[1] & 0xc0) != 0x80) {
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1279 errmsg = "invalid data";
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1280 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1281 endinpos = startinpos+2;
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1282 goto utf8Error;
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1283 }
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1284 ch = ((s[0] & 0x1f) << 6) + (s[1] & 0x3f);
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1285 if (ch < 0x80) {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1286 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1287 endinpos = startinpos+2;
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1288 errmsg = "illegal encoding";
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1289 goto utf8Error;
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1290 }
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1291 else
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1292 *p++ = (Py_UNICODE)ch;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1293 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1294
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1295 case 3:
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	1296 if ((s[1] & 0xc0) != 0x80 \|\|
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1297 (s[2] & 0xc0) != 0x80) {
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1298 errmsg = "invalid data";
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1299 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1300 endinpos = startinpos+3;
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1301 goto utf8Error;
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1302 }
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1303 ch = ((s[0] & 0x0f) << 12) + ((s[1] & 0x3f) << 6) + (s[2] & 0x3f);
21886 abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings. Marc-André Lemburg <mal@egenix.com> parents: 21884 diff changeset	1304 if (ch < 0x0800) {
abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings. Marc-André Lemburg <mal@egenix.com> parents: 21884 diff changeset	1305 /* Note: UTF-8 encodings of surrogates are considered
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	1306 legal UTF-8 sequences;
21886 abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings. Marc-André Lemburg <mal@egenix.com> parents: 21884 diff changeset	1307
abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings. Marc-André Lemburg <mal@egenix.com> parents: 21884 diff changeset	1308 XXX For wide builds (UCS-4) we should probably try
abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings. Marc-André Lemburg <mal@egenix.com> parents: 21884 diff changeset	1309 to recombine the surrogates into a single code
abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings. Marc-André Lemburg <mal@egenix.com> parents: 21884 diff changeset	1310 unit.
abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings. Marc-André Lemburg <mal@egenix.com> parents: 21884 diff changeset	1311 */
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1312 errmsg = "illegal encoding";
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1313 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1314 endinpos = startinpos+3;
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1315 goto utf8Error;
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1316 }
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1317 else
21886 abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings. Marc-André Lemburg <mal@egenix.com> parents: 21884 diff changeset	1318 *p++ = (Py_UNICODE)ch;
13411 9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1319 break;
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1320
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1321 case 4:
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1322 if ((s[1] & 0xc0) != 0x80 \|\|
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1323 (s[2] & 0xc0) != 0x80 \|\|
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1324 (s[3] & 0xc0) != 0x80) {
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1325 errmsg = "invalid data";
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1326 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1327 endinpos = startinpos+4;
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1328 goto utf8Error;
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1329 }
13411 9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1330 ch = ((s[0] & 0x7) << 18) + ((s[1] & 0x3f) << 12) +
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1331 ((s[2] & 0x3f) << 6) + (s[3] & 0x3f);
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1332 /* validate and convert to UTF-16 */
18292 ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1333 if ((ch < 0x10000) /* minimum value allowed for 4
21886 abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings. Marc-André Lemburg <mal@egenix.com> parents: 21884 diff changeset	1334 byte encoding */
18292 ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1335 \|\| (ch > 0x10ffff)) /* maximum value allowed for
21886 abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings. Marc-André Lemburg <mal@egenix.com> parents: 21884 diff changeset	1336 UTF-16 */
18292 ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1337 {
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1338 errmsg = "illegal encoding";
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1339 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1340 endinpos = startinpos+4;
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1341 goto utf8Error;
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1342 }
18324 3ba7a358ce02 use Py_UNICODE_WIDE instead of USE_UCS4_STORAGE and Py_UNICODE_SIZE Fredrik Lundh <fredrik@pythonware.com> parents: 18298 diff changeset	1343 #ifdef Py_UNICODE_WIDE
18292 ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1344 *p++ = (Py_UNICODE)ch;
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1345 #else
13411 9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1346 /* compute and append the two surrogates: */
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	1347
13411 9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1348 /* translate from 10000..10FFFF to 0..FFFF */
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1349 ch -= 0x10000;
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	1350
13411 9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1351 /* high surrogate = top 10 bits added to D800 */
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1352 *p++ = (Py_UNICODE)(0xD800 + (ch >> 10));
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	1353
13411 9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1354 /* low surrogate = bottom 10 bits added to DC00 */
18279 8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case Fredrik Lundh <fredrik@pythonware.com> parents: 18278 diff changeset	1355 *p++ = (Py_UNICODE)(0xDC00 + (ch & 0x03FF));
18292 ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1356 #endif
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1357 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1358
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1359 default:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1360 /* Other sizes are only needed for UCS-4 */
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1361 errmsg = "unsupported Unicode code range";
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1362 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1363 endinpos = startinpos+n;
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1364 goto utf8Error;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1365 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1366 s += n;
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1367 continue;
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	1368
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1369 utf8Error:
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1370 outpos = p-PyUnicode_AS_UNICODE(unicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1371 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1372 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1373 "utf8", errmsg,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1374 starts, size, &startinpos, &endinpos, &exc, &s,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1375 (PyObject **)&unicode, &outpos, &p))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1376 goto onError;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1377 }
33428 57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 33311 diff changeset	1378 if (consumed)
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 33311 diff changeset	1379 *consumed = s-starts;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1380
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1381 /* Adjust length */
30431 cd0c3f916b34 Change checks of PyUnicode_Resize() return value for clarity. Jeremy Hylton <jeremy@alum.mit.edu> parents: 30216 diff changeset	1382 if (_PyUnicode_Resize(&unicode, p - unicode->str) < 0)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1383 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1384
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1385 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1386 Py_XDECREF(exc);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1387 return (PyObject *)unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1388
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1389 onError:
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1390 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1391 Py_XDECREF(exc);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1392 Py_DECREF(unicode);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1393 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1394 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1395
23294 b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1396 /* Allocation strategy: if the string is short, convert into a stack buffer
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1397 and allocate exactly as much space needed at the end. Else allocate the
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1398 maximum possible needed (4 result bytes per Unicode character), and return
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1399 the excess memory at the end.
23203 583c226afde6 Back out 2.140. Martin v. Löwis <martin@v.loewis.de> parents: 23198 diff changeset	1400 */
23198 950a22eb8cc8 PyUnicode_EncodeUTF8: squash compiler wng. The difference of two Tim Peters <tim.peters@gmail.com> parents: 23192 diff changeset	1401 PyObject *
950a22eb8cc8 PyUnicode_EncodeUTF8: squash compiler wng. The difference of two Tim Peters <tim.peters@gmail.com> parents: 23192 diff changeset	1402 PyUnicode_EncodeUTF8(const Py_UNICODE *s,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	1403 Py_ssize_t size,
23198 950a22eb8cc8 PyUnicode_EncodeUTF8: squash compiler wng. The difference of two Tim Peters <tim.peters@gmail.com> parents: 23192 diff changeset	1404 const char *errors)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1405 {
23294 b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1406 #define MAX_SHORT_UNICHARS 300 /* largest size we'll do on the stack */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1407
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	1408 Py_ssize_t i; /* index into s of next input byte */
23294 b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1409 PyObject v; / result string object */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1410 char p; / next free byte in output buffer */
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	1411 Py_ssize_t nallocated; /* number of result bytes allocated */
37662 0678ce9fb6b2 Change more ints to Py_ssize_t. Martin v. Löwis <martin@v.loewis.de> parents: 37661 diff changeset	1412 Py_ssize_t nneeded; /* number of result bytes needed */
23294 b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1413 char stackbuf[MAX_SHORT_UNICHARS * 4];
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1414
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1415 assert(s != NULL);
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1416 assert(size >= 0);
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1417
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1418 if (size <= MAX_SHORT_UNICHARS) {
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1419 /* Write into the stack buffer; nallocated can't overflow.
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1420 * At the end, we'll allocate exactly as much heap space as it
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1421 * turns out we need.
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1422 */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1423 nallocated = Py_SAFE_DOWNCAST(sizeof(stackbuf), size_t, int);
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1424 v = NULL; /* will allocate after we're done */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1425 p = stackbuf;
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1426 }
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1427 else {
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1428 /* Overallocate on the heap, and give the excess back at the end. */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1429 nallocated = size * 4;
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1430 if (nallocated / 4 != size) /* overflow! */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1431 return PyErr_NoMemory();
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1432 v = PyString_FromStringAndSize(NULL, nallocated);
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1433 if (v == NULL)
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1434 return NULL;
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1435 p = PyString_AS_STRING(v);
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1436 }
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1437
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1438 for (i = 0; i < size;) {
13411 9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1439 Py_UCS4 ch = s[i++];
21882 94bac88f150f Fix for the UTF-8 memory allocation bug and the UTF-8 encoding Marc-André Lemburg <mal@egenix.com> parents: 21262 diff changeset	1440
23203 583c226afde6 Back out 2.140. Martin v. Löwis <martin@v.loewis.de> parents: 23198 diff changeset	1441 if (ch < 0x80)
23294 b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1442 /* Encode ASCII */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1443 *p++ = (char) ch;
21882 94bac88f150f Fix for the UTF-8 memory allocation bug and the UTF-8 encoding Marc-André Lemburg <mal@egenix.com> parents: 21262 diff changeset	1444
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1445 else if (ch < 0x0800) {
23294 b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1446 /* Encode Latin-1 */
21884 04cf4e13e5c9 Cosmetics. Marc-André Lemburg <mal@egenix.com> parents: 21883 diff changeset	1447 *p++ = (char)(0xc0 \| (ch >> 6));
04cf4e13e5c9 Cosmetics. Marc-André Lemburg <mal@egenix.com> parents: 21883 diff changeset	1448 *p++ = (char)(0x80 \| (ch & 0x3f));
13411 9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1449 }
21882 94bac88f150f Fix for the UTF-8 memory allocation bug and the UTF-8 encoding Marc-André Lemburg <mal@egenix.com> parents: 21262 diff changeset	1450 else {
23294 b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1451 /* Encode UCS2 Unicode ordinals */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1452 if (ch < 0x10000) {
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1453 /* Special case: check for high surrogate */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1454 if (0xD800 <= ch && ch <= 0xDBFF && i != size) {
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1455 Py_UCS4 ch2 = s[i];
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1456 /* Check for low surrogate and combine the two to
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1457 form a UCS4 value */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1458 if (0xDC00 <= ch2 && ch2 <= 0xDFFF) {
23203 583c226afde6 Back out 2.140. Martin v. Löwis <martin@v.loewis.de> parents: 23198 diff changeset	1459 ch = ((ch - 0xD800) << 10 \| (ch2 - 0xDC00)) + 0x10000;
23294 b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1460 i++;
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1461 goto encodeUCS4;
13411 9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1462 }
23294 b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1463 /* Fall through: handles isolated high surrogates */
13411 9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1464 }
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1465 *p++ = (char)(0xe0 \| (ch >> 12));
23294 b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1466 *p++ = (char)(0x80 \| ((ch >> 6) & 0x3f));
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1467 *p++ = (char)(0x80 \| (ch & 0x3f));
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1468 continue;
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1469 }
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1470 encodeUCS4:
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1471 /* Encode UCS4 Unicode ordinals */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1472 *p++ = (char)(0xf0 \| (ch >> 18));
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1473 *p++ = (char)(0x80 \| ((ch >> 12) & 0x3f));
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1474 *p++ = (char)(0x80 \| ((ch >> 6) & 0x3f));
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1475 *p++ = (char)(0x80 \| (ch & 0x3f));
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1476 }
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1477 }
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1478
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1479 if (v == NULL) {
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1480 /* This was stack allocated. */
37662 0678ce9fb6b2 Change more ints to Py_ssize_t. Martin v. Löwis <martin@v.loewis.de> parents: 37661 diff changeset	1481 nneeded = p - stackbuf;
23294 b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1482 assert(nneeded <= nallocated);
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1483 v = PyString_FromStringAndSize(stackbuf, nneeded);
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1484 }
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1485 else {
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1486 /* Cut back to size actually needed. */
37662 0678ce9fb6b2 Change more ints to Py_ssize_t. Martin v. Löwis <martin@v.loewis.de> parents: 37661 diff changeset	1487 nneeded = p - PyString_AS_STRING(v);
23294 b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1488 assert(nneeded <= nallocated);
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1489 _PyString_Resize(&v, nneeded);
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1490 }
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1491 return v;
23203 583c226afde6 Back out 2.140. Martin v. Löwis <martin@v.loewis.de> parents: 23198 diff changeset	1492
23294 b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1493 #undef MAX_SHORT_UNICHARS
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1494 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1495
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1496 PyObject PyUnicode_AsUTF8String(PyObject unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1497 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1498 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1499 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1500 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1501 }
14059 d0348a26b093 PyUnicode_AsUTF8String(): Don't need to explicitly incref str since Barry Warsaw <barry@python.org> parents: 13966 diff changeset	1502 return PyUnicode_EncodeUTF8(PyUnicode_AS_UNICODE(unicode),
d0348a26b093 PyUnicode_AsUTF8String(): Don't need to explicitly incref str since Barry Warsaw <barry@python.org> parents: 13966 diff changeset	1503 PyUnicode_GET_SIZE(unicode),
d0348a26b093 PyUnicode_AsUTF8String(): Don't need to explicitly incref str since Barry Warsaw <barry@python.org> parents: 13966 diff changeset	1504 NULL);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1505 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1506
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1507 /* --- UTF-16 Codec ------------------------------------------------------- */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1508
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1509 PyObject *
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1510 PyUnicode_DecodeUTF16(const char *s,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	1511 Py_ssize_t size,
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1512 const char *errors,
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1513 int *byteorder)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1514 {
33428 57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 33311 diff changeset	1515 return PyUnicode_DecodeUTF16Stateful(s, size, errors, byteorder, NULL);
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 33311 diff changeset	1516 }
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 33311 diff changeset	1517
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 33311 diff changeset	1518 PyObject *
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 33311 diff changeset	1519 PyUnicode_DecodeUTF16Stateful(const char *s,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	1520 Py_ssize_t size,
33428 57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 33311 diff changeset	1521 const char *errors,
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 33311 diff changeset	1522 int *byteorder,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	1523 Py_ssize_t *consumed)
33428 57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 33311 diff changeset	1524 {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1525 const char *starts = s;
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	1526 Py_ssize_t startinpos;
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	1527 Py_ssize_t endinpos;
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	1528 Py_ssize_t outpos;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1529 PyUnicodeObject *unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1530 Py_UNICODE *p;
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1531 const unsigned char q, e;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1532 int bo = 0; /* assume native ordering by default */
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1533 const char *errmsg = "";
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1534 /* Offsets from q for retrieving byte pairs in the right order. */
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1535 #ifdef BYTEORDER_IS_LITTLE_ENDIAN
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1536 int ihi = 1, ilo = 0;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1537 #else
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1538 int ihi = 0, ilo = 1;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1539 #endif
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1540 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1541 PyObject *exc = NULL;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1542
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1543 /* Note: size will always be longer than the resulting Unicode
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1544 character count */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1545 unicode = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1546 if (!unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1547 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1548 if (size == 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1549 return (PyObject *)unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1550
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1551 /* Unpack UTF-16 encoded data */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1552 p = unicode->str;
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1553 q = (unsigned char *)s;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1554 e = q + size;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1555
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1556 if (byteorder)
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1557 bo = *byteorder;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1558
17887 fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the Marc-André Lemburg <mal@egenix.com> parents: 17700 diff changeset	1559 /* Check for BOM marks (U+FEFF) in the input and adjust current
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the Marc-André Lemburg <mal@egenix.com> parents: 17700 diff changeset	1560 byte order setting accordingly. In native mode, the leading BOM
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the Marc-André Lemburg <mal@egenix.com> parents: 17700 diff changeset	1561 mark is skipped, in all other modes, it is copied to the output
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the Marc-André Lemburg <mal@egenix.com> parents: 17700 diff changeset	1562 stream as-is (giving a ZWNBSP character). */
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the Marc-André Lemburg <mal@egenix.com> parents: 17700 diff changeset	1563 if (bo == 0) {
33428 57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 33311 diff changeset	1564 if (size >= 2) {
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 33311 diff changeset	1565 const Py_UNICODE bom = (q[ihi] << 8) \| q[ilo];
17887 fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the Marc-André Lemburg <mal@egenix.com> parents: 17700 diff changeset	1566 #ifdef BYTEORDER_IS_LITTLE_ENDIAN
33428 57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 33311 diff changeset	1567 if (bom == 0xFEFF) {
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 33311 diff changeset	1568 q += 2;
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 33311 diff changeset	1569 bo = -1;
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 33311 diff changeset	1570 }
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 33311 diff changeset	1571 else if (bom == 0xFFFE) {
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 33311 diff changeset	1572 q += 2;
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 33311 diff changeset	1573 bo = 1;
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 33311 diff changeset	1574 }
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 33311 diff changeset	1575 #else
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 33311 diff changeset	1576 if (bom == 0xFEFF) {
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 33311 diff changeset	1577 q += 2;
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 33311 diff changeset	1578 bo = 1;
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 33311 diff changeset	1579 }
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 33311 diff changeset	1580 else if (bom == 0xFFFE) {
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 33311 diff changeset	1581 q += 2;
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 33311 diff changeset	1582 bo = -1;
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 33311 diff changeset	1583 }
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 33311 diff changeset	1584 #endif
17887 fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the Marc-André Lemburg <mal@egenix.com> parents: 17700 diff changeset	1585 }
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the Marc-André Lemburg <mal@egenix.com> parents: 17700 diff changeset	1586 }
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1587
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1588 if (bo == -1) {
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1589 /* force LE */
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1590 ihi = 1;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1591 ilo = 0;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1592 }
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1593 else if (bo == 1) {
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1594 /* force BE */
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1595 ihi = 0;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1596 ilo = 1;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1597 }
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1598
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1599 while (q < e) {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1600 Py_UNICODE ch;
33428 57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 33311 diff changeset	1601 /* remaining bytes at the end? (size should be even) */
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1602 if (e-q<2) {
33428 57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 33311 diff changeset	1603 if (consumed)
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 33311 diff changeset	1604 break;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1605 errmsg = "truncated data";
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1606 startinpos = ((const char *)q)-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1607 endinpos = ((const char *)e)-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1608 goto utf16Error;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1609 /* The remaining input chars are ignored if the callback
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1610 chooses to skip the input */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1611 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1612 ch = (q[ihi] << 8) \| q[ilo];
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1613
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1614 q += 2;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1615
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1616 if (ch < 0xD800 \|\| ch > 0xDFFF) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1617 *p++ = ch;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1618 continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1619 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1620
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1621 /* UTF-16 code pair: */
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1622 if (q >= e) {
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1623 errmsg = "unexpected end of data";
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1624 startinpos = (((const char *)q)-2)-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1625 endinpos = ((const char *)e)-starts;
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1626 goto utf16Error;
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1627 }
18295 20e3a6e10c24 When decoding UTF-16, don't assume that the buffer is in native endianness Martin v. Löwis <martin@v.loewis.de> parents: 18292 diff changeset	1628 if (0xD800 <= ch && ch <= 0xDBFF) {
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1629 Py_UNICODE ch2 = (q[ihi] << 8) \| q[ilo];
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1630 q += 2;
18295 20e3a6e10c24 When decoding UTF-16, don't assume that the buffer is in native endianness Martin v. Löwis <martin@v.loewis.de> parents: 18292 diff changeset	1631 if (0xDC00 <= ch2 && ch2 <= 0xDFFF) {
18324 3ba7a358ce02 use Py_UNICODE_WIDE instead of USE_UCS4_STORAGE and Py_UNICODE_SIZE Fredrik Lundh <fredrik@pythonware.com> parents: 18298 diff changeset	1632 #ifndef Py_UNICODE_WIDE
18774 57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1633 *p++ = ch;
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1634 *p++ = ch2;
18292 ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1635 #else
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1636 *p++ = (((ch & 0x3FF)<<10) \| (ch2 & 0x3FF)) + 0x10000;
18774 57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1637 #endif
18292 ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1638 continue;
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1639 }
18292 ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1640 else {
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1641 errmsg = "illegal UTF-16 surrogate";
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1642 startinpos = (((const char *)q)-4)-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1643 endinpos = startinpos+2;
18292 ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1644 goto utf16Error;
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1645 }
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1646
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1647 }
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1648 errmsg = "illegal encoding";
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1649 startinpos = (((const char *)q)-2)-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1650 endinpos = startinpos+2;
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1651 /* Fall through to report the error */
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1652
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1653 utf16Error:
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1654 outpos = p-PyUnicode_AS_UNICODE(unicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1655 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1656 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1657 "utf16", errmsg,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1658 starts, size, &startinpos, &endinpos, &exc, (const char **)&q,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1659 (PyObject **)&unicode, &outpos, &p))
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1660 goto onError;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1661 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1662
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1663 if (byteorder)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1664 *byteorder = bo;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1665
33428 57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 33311 diff changeset	1666 if (consumed)
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 33311 diff changeset	1667 consumed = (const char )q-starts;
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 33311 diff changeset	1668
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1669 /* Adjust length */
30431 cd0c3f916b34 Change checks of PyUnicode_Resize() return value for clarity. Jeremy Hylton <jeremy@alum.mit.edu> parents: 30216 diff changeset	1670 if (_PyUnicode_Resize(&unicode, p - unicode->str) < 0)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1671 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1672
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1673 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1674 Py_XDECREF(exc);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1675 return (PyObject *)unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1676
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1677 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1678 Py_DECREF(unicode);
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1679 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1680 Py_XDECREF(exc);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1681 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1682 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1683
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1684 PyObject *
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1685 PyUnicode_EncodeUTF16(const Py_UNICODE *s,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	1686 Py_ssize_t size,
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1687 const char *errors,
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1688 int byteorder)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1689 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1690 PyObject *v;
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1691 unsigned char *p;
31238 4edf200a8118 SF #859573: Reduce compiler warnings on gcc 3.2 and above. Hye-Shik Chang <hyeshik@gmail.com> parents: 31199 diff changeset	1692 #ifdef Py_UNICODE_WIDE
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1693 int i, pairs;
31238 4edf200a8118 SF #859573: Reduce compiler warnings on gcc 3.2 and above. Hye-Shik Chang <hyeshik@gmail.com> parents: 31199 diff changeset	1694 #else
4edf200a8118 SF #859573: Reduce compiler warnings on gcc 3.2 and above. Hye-Shik Chang <hyeshik@gmail.com> parents: 31199 diff changeset	1695 const int pairs = 0;
4edf200a8118 SF #859573: Reduce compiler warnings on gcc 3.2 and above. Hye-Shik Chang <hyeshik@gmail.com> parents: 31199 diff changeset	1696 #endif
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1697 /* Offsets from p for storing byte pairs in the right order. */
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1698 #ifdef BYTEORDER_IS_LITTLE_ENDIAN
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1699 int ihi = 1, ilo = 0;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1700 #else
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1701 int ihi = 0, ilo = 1;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1702 #endif
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1703
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1704 #define STORECHAR(CH) \
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1705 do { \
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1706 p[ihi] = ((CH) >> 8) & 0xff; \
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1707 p[ilo] = (CH) & 0xff; \
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1708 p += 2; \
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1709 } while(0)
18292 ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1710
31238 4edf200a8118 SF #859573: Reduce compiler warnings on gcc 3.2 and above. Hye-Shik Chang <hyeshik@gmail.com> parents: 31199 diff changeset	1711 #ifdef Py_UNICODE_WIDE
18292 ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1712 for (i = pairs = 0; i < size; i++)
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1713 if (s[i] >= 0x10000)
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1714 pairs++;
31238 4edf200a8118 SF #859573: Reduce compiler warnings on gcc 3.2 and above. Hye-Shik Chang <hyeshik@gmail.com> parents: 31199 diff changeset	1715 #endif
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	1716 v = PyString_FromStringAndSize(NULL,
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1717 2 * (size + pairs + (byteorder == 0)));
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1718 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1719 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1720
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1721 p = (unsigned char *)PyString_AS_STRING(v);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1722 if (byteorder == 0)
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1723 STORECHAR(0xFEFF);
13396 759f698740cc Fix to bug #393 (UTF16 codec didn't like empty strings) and Marc-André Lemburg <mal@egenix.com> parents: 13394 diff changeset	1724 if (size == 0)
13966 91ddeed12fa7 Fixed a couple of instances where a 0-length string was being Marc-André Lemburg <mal@egenix.com> parents: 13946 diff changeset	1725 return v;
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1726
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1727 if (byteorder == -1) {
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1728 /* force LE */
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1729 ihi = 1;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1730 ilo = 0;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1731 }
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1732 else if (byteorder == 1) {
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1733 /* force BE */
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1734 ihi = 0;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1735 ilo = 1;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1736 }
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1737
18292 ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1738 while (size-- > 0) {
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1739 Py_UNICODE ch = *s++;
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1740 Py_UNICODE ch2 = 0;
31238 4edf200a8118 SF #859573: Reduce compiler warnings on gcc 3.2 and above. Hye-Shik Chang <hyeshik@gmail.com> parents: 31199 diff changeset	1741 #ifdef Py_UNICODE_WIDE
18292 ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1742 if (ch >= 0x10000) {
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1743 ch2 = 0xDC00 \| ((ch-0x10000) & 0x3FF);
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1744 ch = 0xD800 \| ((ch-0x10000) >> 10);
18292 ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1745 }
31238 4edf200a8118 SF #859573: Reduce compiler warnings on gcc 3.2 and above. Hye-Shik Chang <hyeshik@gmail.com> parents: 31199 diff changeset	1746 #endif
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1747 STORECHAR(ch);
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1748 if (ch2)
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1749 STORECHAR(ch2);
18292 ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1750 }
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1751 return v;
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1752 #undef STORECHAR
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1753 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1754
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1755 PyObject PyUnicode_AsUTF16String(PyObject unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1756 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1757 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1758 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1759 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1760 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1761 return PyUnicode_EncodeUTF16(PyUnicode_AS_UNICODE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1762 PyUnicode_GET_SIZE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1763 NULL,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1764 0);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1765 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1766
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1767 /* --- Unicode Escape Codec ----------------------------------------------- */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1768
16200 3484670c37f2 Move uchhash functionality into unicodedata (after the recent Fredrik Lundh <fredrik@pythonware.com> parents: 16064 diff changeset	1769 static _PyUnicode_Name_CAPI *ucnhash_CAPI = NULL;
13063 eb243443b884 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13049 diff changeset	1770
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1771 PyObject PyUnicode_DecodeUnicodeEscape(const char s,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	1772 Py_ssize_t size,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1773 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1774 {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1775 const char *starts = s;
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	1776 Py_ssize_t startinpos;
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	1777 Py_ssize_t endinpos;
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	1778 Py_ssize_t outpos;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1779 int i;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1780 PyUnicodeObject *v;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1781 Py_UNICODE *p;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1782 const char *end;
16667 725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1783 char* message;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1784 Py_UCS4 chr = 0xffffffff; /* in case 'getcode' messes up */
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1785 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1786 PyObject *exc = NULL;
16667 725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1787
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1788 /* Escaped strings will always be longer than the resulting
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1789 Unicode string, so we start with size here and then reduce the
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1790 length after conversion to the true value.
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1791 (but if the error callback returns a long replacement string
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1792 we'll have to allocate more space) */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1793 v = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1794 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1795 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1796 if (size == 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1797 return (PyObject *)v;
16667 725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1798
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1799 p = PyUnicode_AS_UNICODE(v);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1800 end = s + size;
16667 725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1801
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1802 while (s < end) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1803 unsigned char c;
13396 759f698740cc Fix to bug #393 (UTF16 codec didn't like empty strings) and Marc-André Lemburg <mal@egenix.com> parents: 13394 diff changeset	1804 Py_UNICODE x;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1805 int digits;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1806
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1807 /* Non-escape characters are interpreted as Unicode ordinals */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1808 if (*s != '\\') {
16667 725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1809 p++ = (unsigned char) s++;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1810 continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1811 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1812
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1813 startinpos = s-starts;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1814 /* \ - Escapes */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1815 s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1816 switch (*s++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1817
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1818 /* \x escapes */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1819 case '\n': break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1820 case '\\': *p++ = '\\'; break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1821 case '\'': *p++ = '\''; break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1822 case '\"': *p++ = '\"'; break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1823 case 'b': *p++ = '\b'; break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1824 case 'f': p++ = '\014'; break; / FF */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1825 case 't': *p++ = '\t'; break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1826 case 'n': *p++ = '\n'; break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1827 case 'r': *p++ = '\r'; break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1828 case 'v': p++ = '\013'; break; / VT */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1829 case 'a': p++ = '\007'; break; / BEL, not classic C */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1830
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1831 /* \OOO (octal) escapes */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1832 case '0': case '1': case '2': case '3':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1833 case '4': case '5': case '6': case '7':
12544 599bfa4ffd49 Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12511 diff changeset	1834 x = s[-1] - '0';
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1835 if ('0' <= s && s <= '7') {
12544 599bfa4ffd49 Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12511 diff changeset	1836 x = (x<<3) + *s++ - '0';
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1837 if ('0' <= s && s <= '7')
12544 599bfa4ffd49 Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12511 diff changeset	1838 x = (x<<3) + *s++ - '0';
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1839 }
12544 599bfa4ffd49 Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12511 diff changeset	1840 *p++ = x;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1841 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1842
16667 725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1843 /* hex escapes */
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1844 /* \xXX */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1845 case 'x':
16667 725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1846 digits = 2;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1847 message = "truncated \\xXX escape";
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1848 goto hexescape;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1849
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1850 /* \uXXXX */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1851 case 'u':
16667 725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1852 digits = 4;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1853 message = "truncated \\uXXXX escape";
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1854 goto hexescape;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1855
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1856 /* \UXXXXXXXX */
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1857 case 'U':
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1858 digits = 8;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1859 message = "truncated \\UXXXXXXXX escape";
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1860 hexescape:
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1861 chr = 0;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1862 outpos = p-PyUnicode_AS_UNICODE(v);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1863 if (s+digits>end) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1864 endinpos = size;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1865 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1866 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1867 "unicodeescape", "end of string in escape sequence",
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1868 starts, size, &startinpos, &endinpos, &exc, &s,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1869 (PyObject **)&v, &outpos, &p))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1870 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1871 goto nextByte;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1872 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1873 for (i = 0; i < digits; ++i) {
16667 725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1874 c = (unsigned char) s[i];
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1875 if (!isxdigit(c)) {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1876 endinpos = (s+i+1)-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1877 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1878 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1879 "unicodeescape", message,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1880 starts, size, &startinpos, &endinpos, &exc, &s,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1881 (PyObject **)&v, &outpos, &p))
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1882 goto onError;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1883 goto nextByte;
14388 ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1884 }
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1885 chr = (chr<<4) & ~0xF;
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1886 if (c >= '0' && c <= '9')
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1887 chr += c - '0';
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1888 else if (c >= 'a' && c <= 'f')
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1889 chr += 10 + c - 'a';
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1890 else
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1891 chr += 10 + c - 'A';
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1892 }
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1893 s += i;
30653 3b47d6057151 Fix for SF bug [ 817156 ] invalid \U escape gives 0=length unistr. Jeremy Hylton <jeremy@alum.mit.edu> parents: 30441 diff changeset	1894 if (chr == 0xffffffff && PyErr_Occurred())
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1895 /* _decoding_error will have already written into the
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1896 target buffer. */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1897 break;
16667 725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1898 store:
14388 ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1899 /* when we get here, chr is a 32-bit unicode character */
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1900 if (chr <= 0xffff)
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1901 /* UCS-2 character */
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1902 *p++ = (Py_UNICODE) chr;
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1903 else if (chr <= 0x10ffff) {
18774 57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1904 /* UCS-4 character. Either store directly, or as
22521 795bdd692176 Fix whitespace. Walter Dörwald <walter@livinglogic.de> parents: 22446 diff changeset	1905 surrogate pair. */
18324 3ba7a358ce02 use Py_UNICODE_WIDE instead of USE_UCS4_STORAGE and Py_UNICODE_SIZE Fredrik Lundh <fredrik@pythonware.com> parents: 18298 diff changeset	1906 #ifdef Py_UNICODE_WIDE
18292 ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1907 *p++ = chr;
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1908 #else
14388 ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1909 chr -= 0x10000L;
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1910 *p++ = 0xD800 + (Py_UNICODE) (chr >> 10);
18279 8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case Fredrik Lundh <fredrik@pythonware.com> parents: 18278 diff changeset	1911 *p++ = 0xDC00 + (Py_UNICODE) (chr & 0x03FF);
18292 ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1912 #endif
14388 ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1913 } else {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1914 endinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1915 outpos = p-PyUnicode_AS_UNICODE(v);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1916 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1917 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1918 "unicodeescape", "illegal Unicode character",
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1919 starts, size, &startinpos, &endinpos, &exc, &s,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1920 (PyObject **)&v, &outpos, &p))
14388 ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1921 goto onError;
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1922 }
16667 725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1923 break;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1924
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1925 /* \N{name} */
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1926 case 'N':
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1927 message = "malformed \\N character escape";
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1928 if (ucnhash_CAPI == NULL) {
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1929 /* load the unicode data module */
36877 e9a7a2784e17 SF #1444030: Fix several potential defects found by Coverity. Hye-Shik Chang <hyeshik@gmail.com> parents: 36675 diff changeset	1930 PyObject m, api;
16667 725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1931 m = PyImport_ImportModule("unicodedata");
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1932 if (m == NULL)
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1933 goto ucnhashError;
36877 e9a7a2784e17 SF #1444030: Fix several potential defects found by Coverity. Hye-Shik Chang <hyeshik@gmail.com> parents: 36675 diff changeset	1934 api = PyObject_GetAttrString(m, "ucnhash_CAPI");
16667 725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1935 Py_DECREF(m);
36877 e9a7a2784e17 SF #1444030: Fix several potential defects found by Coverity. Hye-Shik Chang <hyeshik@gmail.com> parents: 36675 diff changeset	1936 if (api == NULL)
16667 725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1937 goto ucnhashError;
37596 1b078493d538 More low-hanging fruit. Still need to re-arrange some code (or find a better Anthony Baxter <anthonybaxter@gmail.com> parents: 37306 diff changeset	1938 ucnhash_CAPI = (_PyUnicode_Name_CAPI *)PyCObject_AsVoidPtr(api);
36877 e9a7a2784e17 SF #1444030: Fix several potential defects found by Coverity. Hye-Shik Chang <hyeshik@gmail.com> parents: 36675 diff changeset	1939 Py_DECREF(api);
16667 725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1940 if (ucnhash_CAPI == NULL)
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1941 goto ucnhashError;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1942 }
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1943 if (*s == '{') {
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1944 const char *start = s+1;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1945 /* look for the closing brace */
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1946 while (*s != '}' && s < end)
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1947 s++;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1948 if (s > start && s < end && *s == '}') {
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1949 /* found a name. look it up in the unicode database */
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1950 message = "unknown Unicode character name";
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1951 s++;
36924 48e0a5e39403 Update Unicode database to Unicode 4.1. Martin v. Löwis <martin@v.loewis.de> parents: 36882 diff changeset	1952 if (ucnhash_CAPI->getcode(NULL, start, (int)(s-start-1), &chr))
16667 725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1953 goto store;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1954 }
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1955 }
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1956 endinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1957 outpos = p-PyUnicode_AS_UNICODE(v);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1958 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1959 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1960 "unicodeescape", message,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1961 starts, size, &startinpos, &endinpos, &exc, &s,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1962 (PyObject **)&v, &outpos, &p))
16667 725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1963 goto onError;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1964 break;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1965
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1966 default:
22521 795bdd692176 Fix whitespace. Walter Dörwald <walter@livinglogic.de> parents: 22446 diff changeset	1967 if (s > end) {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1968 message = "\\ at end of string";
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1969 s--;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1970 endinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1971 outpos = p-PyUnicode_AS_UNICODE(v);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1972 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1973 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1974 "unicodeescape", message,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1975 starts, size, &startinpos, &endinpos, &exc, &s,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1976 (PyObject **)&v, &outpos, &p))
22521 795bdd692176 Fix whitespace. Walter Dörwald <walter@livinglogic.de> parents: 22446 diff changeset	1977 goto onError;
795bdd692176 Fix whitespace. Walter Dörwald <walter@livinglogic.de> parents: 22446 diff changeset	1978 }
795bdd692176 Fix whitespace. Walter Dörwald <walter@livinglogic.de> parents: 22446 diff changeset	1979 else {
795bdd692176 Fix whitespace. Walter Dörwald <walter@livinglogic.de> parents: 22446 diff changeset	1980 *p++ = '\\';
795bdd692176 Fix whitespace. Walter Dörwald <walter@livinglogic.de> parents: 22446 diff changeset	1981 *p++ = (unsigned char)s[-1];
795bdd692176 Fix whitespace. Walter Dörwald <walter@livinglogic.de> parents: 22446 diff changeset	1982 }
16667 725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1983 break;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1984 }
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1985 nextByte:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1986 ;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1987 }
37662 0678ce9fb6b2 Change more ints to Py_ssize_t. Martin v. Löwis <martin@v.loewis.de> parents: 37661 diff changeset	1988 if (_PyUnicode_Resize(&v, p - PyUnicode_AS_UNICODE(v)) < 0)
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1989 goto onError;
30160 3329b058db20 Fix another refcounting leak (in PyUnicode_DecodeUnicodeEscape()). Walter Dörwald <walter@livinglogic.de> parents: 30149 diff changeset	1990 Py_XDECREF(errorHandler);
3329b058db20 Fix another refcounting leak (in PyUnicode_DecodeUnicodeEscape()). Walter Dörwald <walter@livinglogic.de> parents: 30149 diff changeset	1991 Py_XDECREF(exc);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1992 return (PyObject *)v;
22521 795bdd692176 Fix whitespace. Walter Dörwald <walter@livinglogic.de> parents: 22446 diff changeset	1993
16667 725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1994 ucnhashError:
16200 3484670c37f2 Move uchhash functionality into unicodedata (after the recent Fredrik Lundh <fredrik@pythonware.com> parents: 16064 diff changeset	1995 PyErr_SetString(
3484670c37f2 Move uchhash functionality into unicodedata (after the recent Fredrik Lundh <fredrik@pythonware.com> parents: 16064 diff changeset	1996 PyExc_UnicodeError,
3484670c37f2 Move uchhash functionality into unicodedata (after the recent Fredrik Lundh <fredrik@pythonware.com> parents: 16064 diff changeset	1997 "\\N escapes not supported (can't load unicodedata module)"
3484670c37f2 Move uchhash functionality into unicodedata (after the recent Fredrik Lundh <fredrik@pythonware.com> parents: 16064 diff changeset	1998 );
36877 e9a7a2784e17 SF #1444030: Fix several potential defects found by Coverity. Hye-Shik Chang <hyeshik@gmail.com> parents: 36675 diff changeset	1999 Py_XDECREF(v);
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2000 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2001 Py_XDECREF(exc);
16064 a97aa9238771 Better error message if ucnhash cannot be found (obscure attribute Fredrik Lundh <fredrik@pythonware.com> parents: 16029 diff changeset	2002 return NULL;
a97aa9238771 Better error message if ucnhash cannot be found (obscure attribute Fredrik Lundh <fredrik@pythonware.com> parents: 16029 diff changeset	2003
16667 725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	2004 onError:
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2005 Py_XDECREF(v);
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2006 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2007 Py_XDECREF(exc);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2008 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2009 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2010
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2011 /* Return a Unicode-Escape string version of the Unicode object.
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2012
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2013 If quotes is true, the string is enclosed in u"" or u'' quotes as
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2014 appropriate.
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2015
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2016 */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2017
38440 5b0753b9d16a needforspeed: replace improvements, changed to Py_LOCAL_INLINE Fredrik Lundh <fredrik@pythonware.com> parents: 38427 diff changeset	2018 Py_LOCAL_INLINE(const Py_UNICODE ) findchar(const Py_UNICODE s,
38347 6d8615f9f1cc use Py_LOCAL also for string and unicode objects Fredrik Lundh <fredrik@pythonware.com> parents: 38339 diff changeset	2019 Py_ssize_t size,
6d8615f9f1cc use Py_LOCAL also for string and unicode objects Fredrik Lundh <fredrik@pythonware.com> parents: 38339 diff changeset	2020 Py_UNICODE ch)
38278 260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	2021 {
260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	2022 /* like wcschr, but doesn't stop at NULL characters */
260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	2023
260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	2024 while (size-- > 0) {
260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	2025 if (*s == ch)
260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	2026 return s;
260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	2027 s++;
260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	2028 }
260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	2029
260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	2030 return NULL;
260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	2031 }
12025 7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said: Barry Warsaw <barry@python.org> parents: 11997 diff changeset	2032
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2033 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2034 PyObject unicodeescape_string(const Py_UNICODE s,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	2035 Py_ssize_t size,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2036 int quotes)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2037 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2038 PyObject *repr;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2039 char *p;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2040
16226 c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings. Ka-Ping Yee <ping@zesty.ca> parents: 16200 diff changeset	2041 static const char *hexdigit = "0123456789abcdef";
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2042
39621 e8a65bbd6a29 Patch #1541585: fix buffer overrun when performing repr() on Neal Norwitz <nnorwitz@gmail.com> parents: 39491 diff changeset	2043 /* XXX(nnorwitz): rather than over-allocating, it would be
e8a65bbd6a29 Patch #1541585: fix buffer overrun when performing repr() on Neal Norwitz <nnorwitz@gmail.com> parents: 39491 diff changeset	2044 better to choose a different scheme. Perhaps scan the
e8a65bbd6a29 Patch #1541585: fix buffer overrun when performing repr() on Neal Norwitz <nnorwitz@gmail.com> parents: 39491 diff changeset	2045 first N-chars of the string and allocate based on that size.
e8a65bbd6a29 Patch #1541585: fix buffer overrun when performing repr() on Neal Norwitz <nnorwitz@gmail.com> parents: 39491 diff changeset	2046 */
e8a65bbd6a29 Patch #1541585: fix buffer overrun when performing repr() on Neal Norwitz <nnorwitz@gmail.com> parents: 39491 diff changeset	2047 /* Initial allocation is based on the longest-possible unichr
e8a65bbd6a29 Patch #1541585: fix buffer overrun when performing repr() on Neal Norwitz <nnorwitz@gmail.com> parents: 39491 diff changeset	2048 escape.
e8a65bbd6a29 Patch #1541585: fix buffer overrun when performing repr() on Neal Norwitz <nnorwitz@gmail.com> parents: 39491 diff changeset	2049
e8a65bbd6a29 Patch #1541585: fix buffer overrun when performing repr() on Neal Norwitz <nnorwitz@gmail.com> parents: 39491 diff changeset	2050 In wide (UTF-32) builds '\U00xxxxxx' is 10 chars per source
e8a65bbd6a29 Patch #1541585: fix buffer overrun when performing repr() on Neal Norwitz <nnorwitz@gmail.com> parents: 39491 diff changeset	2051 unichr, so in this case it's the longest unichr escape. In
e8a65bbd6a29 Patch #1541585: fix buffer overrun when performing repr() on Neal Norwitz <nnorwitz@gmail.com> parents: 39491 diff changeset	2052 narrow (UTF-16) builds this is five chars per source unichr
e8a65bbd6a29 Patch #1541585: fix buffer overrun when performing repr() on Neal Norwitz <nnorwitz@gmail.com> parents: 39491 diff changeset	2053 since there are two unichrs in the surrogate pair, so in narrow
e8a65bbd6a29 Patch #1541585: fix buffer overrun when performing repr() on Neal Norwitz <nnorwitz@gmail.com> parents: 39491 diff changeset	2054 (UTF-16) builds it's not the longest unichr escape.
e8a65bbd6a29 Patch #1541585: fix buffer overrun when performing repr() on Neal Norwitz <nnorwitz@gmail.com> parents: 39491 diff changeset	2055
e8a65bbd6a29 Patch #1541585: fix buffer overrun when performing repr() on Neal Norwitz <nnorwitz@gmail.com> parents: 39491 diff changeset	2056 In wide or narrow builds '\uxxxx' is 6 chars per source unichr,
e8a65bbd6a29 Patch #1541585: fix buffer overrun when performing repr() on Neal Norwitz <nnorwitz@gmail.com> parents: 39491 diff changeset	2057 so in the narrow (UTF-16) build case it's the longest unichr
e8a65bbd6a29 Patch #1541585: fix buffer overrun when performing repr() on Neal Norwitz <nnorwitz@gmail.com> parents: 39491 diff changeset	2058 escape.
e8a65bbd6a29 Patch #1541585: fix buffer overrun when performing repr() on Neal Norwitz <nnorwitz@gmail.com> parents: 39491 diff changeset	2059 */
e8a65bbd6a29 Patch #1541585: fix buffer overrun when performing repr() on Neal Norwitz <nnorwitz@gmail.com> parents: 39491 diff changeset	2060
e8a65bbd6a29 Patch #1541585: fix buffer overrun when performing repr() on Neal Norwitz <nnorwitz@gmail.com> parents: 39491 diff changeset	2061 repr = PyString_FromStringAndSize(NULL,
e8a65bbd6a29 Patch #1541585: fix buffer overrun when performing repr() on Neal Norwitz <nnorwitz@gmail.com> parents: 39491 diff changeset	2062 2
e8a65bbd6a29 Patch #1541585: fix buffer overrun when performing repr() on Neal Norwitz <nnorwitz@gmail.com> parents: 39491 diff changeset	2063 #ifdef Py_UNICODE_WIDE
e8a65bbd6a29 Patch #1541585: fix buffer overrun when performing repr() on Neal Norwitz <nnorwitz@gmail.com> parents: 39491 diff changeset	2064 + 10*size
e8a65bbd6a29 Patch #1541585: fix buffer overrun when performing repr() on Neal Norwitz <nnorwitz@gmail.com> parents: 39491 diff changeset	2065 #else
e8a65bbd6a29 Patch #1541585: fix buffer overrun when performing repr() on Neal Norwitz <nnorwitz@gmail.com> parents: 39491 diff changeset	2066 + 6*size
e8a65bbd6a29 Patch #1541585: fix buffer overrun when performing repr() on Neal Norwitz <nnorwitz@gmail.com> parents: 39491 diff changeset	2067 #endif
e8a65bbd6a29 Patch #1541585: fix buffer overrun when performing repr() on Neal Norwitz <nnorwitz@gmail.com> parents: 39491 diff changeset	2068 + 1);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2069 if (repr == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2070 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2071
18828 081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on Marc-André Lemburg <mal@egenix.com> parents: 18774 diff changeset	2072 p = PyString_AS_STRING(repr);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2073
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2074 if (quotes) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2075 *p++ = 'u';
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	2076 *p++ = (findchar(s, size, '\'') &&
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2077 !findchar(s, size, '"')) ? '"' : '\'';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2078 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2079 while (size-- > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2080 Py_UNICODE ch = *s++;
18828 081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on Marc-André Lemburg <mal@egenix.com> parents: 18774 diff changeset	2081
36056 f1ae11457d34 Bug #1379994: Fix unicode_escape codecs to encode r'\' as r'\\' Hye-Shik Chang <hyeshik@gmail.com>* parents: 35999 diff changeset	2082 /* Escape quotes and backslashes */
f1ae11457d34 Bug #1379994: Fix unicode_escape codecs to encode r'\' as r'\\' Hye-Shik Chang <hyeshik@gmail.com>* parents: 35999 diff changeset	2083 if ((quotes &&
f1ae11457d34 Bug #1379994: Fix unicode_escape codecs to encode r'\' as r'\\' Hye-Shik Chang <hyeshik@gmail.com>* parents: 35999 diff changeset	2084 ch == (Py_UNICODE) PyString_AS_STRING(repr)[1]) \|\| ch == '\\') {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2085 *p++ = '\\';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2086 *p++ = (char) ch;
20018 8aa1ada33f64 Fix a bug in rendering of \\ by repr() -- it rendered as \\\ instead Guido van Rossum <guido@python.org> parents: 20007 diff changeset	2087 continue;
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	2088 }
18828 081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on Marc-André Lemburg <mal@egenix.com> parents: 18774 diff changeset	2089
18773 743f8e1d953c #ifdef out generation of \U escapes unless Py_UNICODE_WIDE. This Guido van Rossum <guido@python.org> parents: 18324 diff changeset	2090 #ifdef Py_UNICODE_WIDE
18292 ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	2091 /* Map 21-bit characters to '\U00xxxxxx' */
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	2092 else if (ch >= 0x10000) {
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	2093 *p++ = '\\';
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	2094 *p++ = 'U';
18774 57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	2095 *p++ = hexdigit[(ch >> 28) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	2096 *p++ = hexdigit[(ch >> 24) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	2097 *p++ = hexdigit[(ch >> 20) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	2098 *p++ = hexdigit[(ch >> 16) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	2099 *p++ = hexdigit[(ch >> 12) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	2100 *p++ = hexdigit[(ch >> 8) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	2101 *p++ = hexdigit[(ch >> 4) & 0x0000000F];
18828 081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on Marc-André Lemburg <mal@egenix.com> parents: 18774 diff changeset	2102 *p++ = hexdigit[ch & 0x0000000F];
081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on Marc-André Lemburg <mal@egenix.com> parents: 18774 diff changeset	2103 continue;
18292 ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	2104 }
39621 e8a65bbd6a29 Patch #1541585: fix buffer overrun when performing repr() on Neal Norwitz <nnorwitz@gmail.com> parents: 39491 diff changeset	2105 #else
e8a65bbd6a29 Patch #1541585: fix buffer overrun when performing repr() on Neal Norwitz <nnorwitz@gmail.com> parents: 39491 diff changeset	2106 /* Map UTF-16 surrogate pairs to '\U00xxxxxx' */
18774 57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	2107 else if (ch >= 0xD800 && ch < 0xDC00) {
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	2108 Py_UNICODE ch2;
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	2109 Py_UCS4 ucs;
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	2110
18774 57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	2111 ch2 = *s++;
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	2112 size--;
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	2113 if (ch2 >= 0xDC00 && ch2 <= 0xDFFF) {
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	2114 ucs = (((ch & 0x03FF) << 10) \| (ch2 & 0x03FF)) + 0x00010000;
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	2115 *p++ = '\\';
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	2116 *p++ = 'U';
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	2117 *p++ = hexdigit[(ucs >> 28) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	2118 *p++ = hexdigit[(ucs >> 24) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	2119 *p++ = hexdigit[(ucs >> 20) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	2120 *p++ = hexdigit[(ucs >> 16) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	2121 *p++ = hexdigit[(ucs >> 12) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	2122 *p++ = hexdigit[(ucs >> 8) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	2123 *p++ = hexdigit[(ucs >> 4) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	2124 *p++ = hexdigit[ucs & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	2125 continue;
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	2126 }
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	2127 /* Fall through: isolated surrogates are copied as-is */
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	2128 s--;
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	2129 size++;
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	2130 }
39621 e8a65bbd6a29 Patch #1541585: fix buffer overrun when performing repr() on Neal Norwitz <nnorwitz@gmail.com> parents: 39491 diff changeset	2131 #endif
18774 57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	2132
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2133 /* Map 16-bit characters to '\uxxxx' */
18774 57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	2134 if (ch >= 256) {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2135 *p++ = '\\';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2136 *p++ = 'u';
18774 57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	2137 *p++ = hexdigit[(ch >> 12) & 0x000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	2138 *p++ = hexdigit[(ch >> 8) & 0x000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	2139 *p++ = hexdigit[(ch >> 4) & 0x000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	2140 *p++ = hexdigit[ch & 0x000F];
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2141 }
18828 081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on Marc-André Lemburg <mal@egenix.com> parents: 18774 diff changeset	2142
16226 c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings. Ka-Ping Yee <ping@zesty.ca> parents: 16200 diff changeset	2143 /* Map special whitespace to '\t', \n', '\r' */
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings. Ka-Ping Yee <ping@zesty.ca> parents: 16200 diff changeset	2144 else if (ch == '\t') {
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings. Ka-Ping Yee <ping@zesty.ca> parents: 16200 diff changeset	2145 *p++ = '\\';
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings. Ka-Ping Yee <ping@zesty.ca> parents: 16200 diff changeset	2146 *p++ = 't';
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings. Ka-Ping Yee <ping@zesty.ca> parents: 16200 diff changeset	2147 }
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings. Ka-Ping Yee <ping@zesty.ca> parents: 16200 diff changeset	2148 else if (ch == '\n') {
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings. Ka-Ping Yee <ping@zesty.ca> parents: 16200 diff changeset	2149 *p++ = '\\';
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings. Ka-Ping Yee <ping@zesty.ca> parents: 16200 diff changeset	2150 *p++ = 'n';
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings. Ka-Ping Yee <ping@zesty.ca> parents: 16200 diff changeset	2151 }
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings. Ka-Ping Yee <ping@zesty.ca> parents: 16200 diff changeset	2152 else if (ch == '\r') {
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings. Ka-Ping Yee <ping@zesty.ca> parents: 16200 diff changeset	2153 *p++ = '\\';
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings. Ka-Ping Yee <ping@zesty.ca> parents: 16200 diff changeset	2154 *p++ = 'r';
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings. Ka-Ping Yee <ping@zesty.ca> parents: 16200 diff changeset	2155 }
18828 081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on Marc-André Lemburg <mal@egenix.com> parents: 18774 diff changeset	2156
16226 c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings. Ka-Ping Yee <ping@zesty.ca> parents: 16200 diff changeset	2157 /* Map non-printable US ASCII to '\xhh' */
21067 c9c4c523dcbf Fix for bug #485951: repr diff between string and unicode. Marc-André Lemburg <mal@egenix.com> parents: 21028 diff changeset	2158 else if (ch < ' ' \|\| ch >= 0x7F) {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2159 *p++ = '\\';
16226 c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings. Ka-Ping Yee <ping@zesty.ca> parents: 16200 diff changeset	2160 *p++ = 'x';
18774 57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	2161 *p++ = hexdigit[(ch >> 4) & 0x000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	2162 *p++ = hexdigit[ch & 0x000F];
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	2163 }
18828 081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on Marc-André Lemburg <mal@egenix.com> parents: 18774 diff changeset	2164
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2165 /* Copy everything else as-is */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2166 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2167 *p++ = (char) ch;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2168 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2169 if (quotes)
18828 081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on Marc-André Lemburg <mal@egenix.com> parents: 18774 diff changeset	2170 *p++ = PyString_AS_STRING(repr)[1];
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2171
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2172 *p = '\0';
23295 6fa76b4e59fe Repair widespread misuse of _PyString_Resize. Since it's clear people Tim Peters <tim.peters@gmail.com> parents: 23294 diff changeset	2173 _PyString_Resize(&repr, p - PyString_AS_STRING(repr));
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2174 return repr;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2175 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2176
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2177 PyObject PyUnicode_EncodeUnicodeEscape(const Py_UNICODE s,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	2178 Py_ssize_t size)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2179 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2180 return unicodeescape_string(s, size, 0);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2181 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2182
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2183 PyObject PyUnicode_AsUnicodeEscapeString(PyObject unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2184 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2185 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2186 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2187 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2188 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2189 return PyUnicode_EncodeUnicodeEscape(PyUnicode_AS_UNICODE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2190 PyUnicode_GET_SIZE(unicode));
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2191 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2192
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2193 /* --- Raw Unicode Escape Codec ------------------------------------------- */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2194
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2195 PyObject PyUnicode_DecodeRawUnicodeEscape(const char s,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	2196 Py_ssize_t size,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2197 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2198 {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2199 const char *starts = s;
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	2200 Py_ssize_t startinpos;
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	2201 Py_ssize_t endinpos;
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	2202 Py_ssize_t outpos;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2203 PyUnicodeObject *v;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2204 Py_UNICODE *p;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2205 const char *end;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2206 const char *bs;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2207 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2208 PyObject *exc = NULL;
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	2209
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2210 /* Escaped strings will always be longer than the resulting
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2211 Unicode string, so we start with size here and then reduce the
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2212 length after conversion to the true value. (But decoding error
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2213 handler might have to resize the string) */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2214 v = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2215 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2216 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2217 if (size == 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2218 return (PyObject *)v;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2219 p = PyUnicode_AS_UNICODE(v);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2220 end = s + size;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2221 while (s < end) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2222 unsigned char c;
22423 b523a470a0ce Do not insert characters for unicode-escape decoders if the error mode Martin v. Löwis <martin@v.loewis.de> parents: 22087 diff changeset	2223 Py_UCS4 x;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2224 int i;
29164 4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2225 int count;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2226
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2227 /* Non-escape characters are interpreted as Unicode ordinals */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2228 if (*s != '\\') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2229 p++ = (unsigned char)s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2230 continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2231 }
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2232 startinpos = s-starts;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2233
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2234 /* \u-escapes are only interpreted iff the number of leading
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2235 backslashes if odd */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2236 bs = s;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2237 for (;s < end;) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2238 if (*s != '\\')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2239 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2240 p++ = (unsigned char)s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2241 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2242 if (((s - bs) & 1) == 0 \|\|
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2243 s >= end \|\|
29164 4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2244 (s != 'u' && s != 'U')) {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2245 continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2246 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2247 p--;
29164 4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2248 count = *s=='u' ? 4 : 8;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2249 s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2250
29164 4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2251 /* \uXXXX with 4 hex digits, \Uxxxxxxxx with 8 */
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2252 outpos = p-PyUnicode_AS_UNICODE(v);
29164 4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2253 for (x = 0, i = 0; i < count; ++i, ++s) {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2254 c = (unsigned char)*s;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2255 if (!isxdigit(c)) {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2256 endinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2257 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2258 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2259 "rawunicodeescape", "truncated \\uXXXX",
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2260 starts, size, &startinpos, &endinpos, &exc, &s,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2261 (PyObject **)&v, &outpos, &p))
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2262 goto onError;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2263 goto nextByte;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2264 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2265 x = (x<<4) & ~0xF;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2266 if (c >= '0' && c <= '9')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2267 x += c - '0';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2268 else if (c >= 'a' && c <= 'f')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2269 x += 10 + c - 'a';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2270 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2271 x += 10 + c - 'A';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2272 }
29164 4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2273 #ifndef Py_UNICODE_WIDE
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2274 if (x > 0x10000) {
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2275 if (unicode_decode_call_errorhandler(
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2276 errors, &errorHandler,
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2277 "rawunicodeescape", "\\Uxxxxxxxx out of range",
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2278 starts, size, &startinpos, &endinpos, &exc, &s,
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2279 (PyObject **)&v, &outpos, &p))
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2280 goto onError;
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2281 }
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2282 #endif
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2283 *p++ = x;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2284 nextByte:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2285 ;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2286 }
37662 0678ce9fb6b2 Change more ints to Py_ssize_t. Martin v. Löwis <martin@v.loewis.de> parents: 37661 diff changeset	2287 if (_PyUnicode_Resize(&v, p - PyUnicode_AS_UNICODE(v)) < 0)
12329 3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	2288 goto onError;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2289 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2290 Py_XDECREF(exc);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2291 return (PyObject *)v;
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	2292
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2293 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2294 Py_XDECREF(v);
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2295 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2296 Py_XDECREF(exc);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2297 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2298 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2299
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2300 PyObject PyUnicode_EncodeRawUnicodeEscape(const Py_UNICODE s,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	2301 Py_ssize_t size)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2302 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2303 PyObject *repr;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2304 char *p;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2305 char *q;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2306
16226 c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings. Ka-Ping Yee <ping@zesty.ca> parents: 16200 diff changeset	2307 static const char *hexdigit = "0123456789abcdef";
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2308
29164 4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2309 #ifdef Py_UNICODE_WIDE
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2310 repr = PyString_FromStringAndSize(NULL, 10 * size);
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2311 #else
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2312 repr = PyString_FromStringAndSize(NULL, 6 * size);
29164 4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2313 #endif
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2314 if (repr == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2315 return NULL;
13966 91ddeed12fa7 Fixed a couple of instances where a 0-length string was being Marc-André Lemburg <mal@egenix.com> parents: 13946 diff changeset	2316 if (size == 0)
91ddeed12fa7 Fixed a couple of instances where a 0-length string was being Marc-André Lemburg <mal@egenix.com> parents: 13946 diff changeset	2317 return repr;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2318
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2319 p = q = PyString_AS_STRING(repr);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2320 while (size-- > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2321 Py_UNICODE ch = *s++;
29164 4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2322 #ifdef Py_UNICODE_WIDE
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2323 /* Map 32-bit characters to '\Uxxxxxxxx' */
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2324 if (ch >= 0x10000) {
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2325 *p++ = '\\';
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2326 *p++ = 'U';
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2327 *p++ = hexdigit[(ch >> 28) & 0xf];
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2328 *p++ = hexdigit[(ch >> 24) & 0xf];
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2329 *p++ = hexdigit[(ch >> 20) & 0xf];
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2330 *p++ = hexdigit[(ch >> 16) & 0xf];
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2331 *p++ = hexdigit[(ch >> 12) & 0xf];
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2332 *p++ = hexdigit[(ch >> 8) & 0xf];
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2333 *p++ = hexdigit[(ch >> 4) & 0xf];
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2334 *p++ = hexdigit[ch & 15];
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	2335 }
29164 4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2336 else
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2337 #endif
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2338 /* Map 16-bit characters to '\uxxxx' */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2339 if (ch >= 256) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2340 *p++ = '\\';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2341 *p++ = 'u';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2342 *p++ = hexdigit[(ch >> 12) & 0xf];
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2343 *p++ = hexdigit[(ch >> 8) & 0xf];
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2344 *p++ = hexdigit[(ch >> 4) & 0xf];
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2345 *p++ = hexdigit[ch & 15];
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2346 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2347 /* Copy everything else as-is */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2348 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2349 *p++ = (char) ch;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2350 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2351 *p = '\0';
23295 6fa76b4e59fe Repair widespread misuse of _PyString_Resize. Since it's clear people Tim Peters <tim.peters@gmail.com> parents: 23294 diff changeset	2352 _PyString_Resize(&repr, p - q);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2353 return repr;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2354 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2355
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2356 PyObject PyUnicode_AsRawUnicodeEscapeString(PyObject unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2357 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2358 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2359 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2360 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2361 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2362 return PyUnicode_EncodeRawUnicodeEscape(PyUnicode_AS_UNICODE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2363 PyUnicode_GET_SIZE(unicode));
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2364 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2365
35485 6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2366 /* --- Unicode Internal Codec ------------------------------------------- */
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2367
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2368 PyObject _PyUnicode_DecodeUnicodeInternal(const char s,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	2369 Py_ssize_t size,
35485 6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2370 const char *errors)
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2371 {
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2372 const char *starts = s;
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	2373 Py_ssize_t startinpos;
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	2374 Py_ssize_t endinpos;
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	2375 Py_ssize_t outpos;
35485 6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2376 PyUnicodeObject *v;
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2377 Py_UNICODE *p;
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2378 const char *end;
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2379 const char *reason;
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2380 PyObject *errorHandler = NULL;
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2381 PyObject *exc = NULL;
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2382
36236 ecd423718a32 Fix icc warnings: remove (sometimes) unused variable conditionally Neal Norwitz <nnorwitz@gmail.com> parents: 36203 diff changeset	2383 #ifdef Py_UNICODE_WIDE
ecd423718a32 Fix icc warnings: remove (sometimes) unused variable conditionally Neal Norwitz <nnorwitz@gmail.com> parents: 36203 diff changeset	2384 Py_UNICODE unimax = PyUnicode_GetMax();
ecd423718a32 Fix icc warnings: remove (sometimes) unused variable conditionally Neal Norwitz <nnorwitz@gmail.com> parents: 36203 diff changeset	2385 #endif
ecd423718a32 Fix icc warnings: remove (sometimes) unused variable conditionally Neal Norwitz <nnorwitz@gmail.com> parents: 36203 diff changeset	2386
39995 6ba402b60041 Forward-port of r52136,52138: a review of overflow-detecting code. Armin Rigo <arigo@tunes.org> parents: 39734 diff changeset	2387 /* XXX overflow detection missing */
35485 6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2388 v = _PyUnicode_New((size+Py_UNICODE_SIZE-1)/ Py_UNICODE_SIZE);
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2389 if (v == NULL)
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2390 goto onError;
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2391 if (PyUnicode_GetSize((PyObject *)v) == 0)
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2392 return (PyObject *)v;
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2393 p = PyUnicode_AS_UNICODE(v);
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2394 end = s + size;
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2395
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2396 while (s < end) {
38182 d8355cccff4a Patch #1488312, Fix memory alignment problem on SPARC in unicode. Will backport Neal Norwitz <nnorwitz@gmail.com> parents: 37775 diff changeset	2397 memcpy(p, s, sizeof(Py_UNICODE));
35485 6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2398 /* We have to sanity check the raw data, otherwise doom looms for
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2399 some malformed UCS-4 data. */
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2400 if (
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2401 #ifdef Py_UNICODE_WIDE
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2402 p > unimax \|\| p < 0 \|\|
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2403 #endif
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2404 end-s < Py_UNICODE_SIZE
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2405 )
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2406 {
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2407 startinpos = s - starts;
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2408 if (end-s < Py_UNICODE_SIZE) {
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2409 endinpos = end-starts;
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2410 reason = "truncated input";
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2411 }
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2412 else {
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2413 endinpos = s - starts + Py_UNICODE_SIZE;
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2414 reason = "illegal code point (> 0x10FFFF)";
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2415 }
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2416 outpos = p - PyUnicode_AS_UNICODE(v);
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2417 if (unicode_decode_call_errorhandler(
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2418 errors, &errorHandler,
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2419 "unicode_internal", reason,
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2420 starts, size, &startinpos, &endinpos, &exc, &s,
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2421 (PyObject **)&v, &outpos, &p)) {
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2422 goto onError;
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2423 }
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2424 }
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2425 else {
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2426 p++;
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2427 s += Py_UNICODE_SIZE;
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2428 }
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2429 }
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2430
37662 0678ce9fb6b2 Change more ints to Py_ssize_t. Martin v. Löwis <martin@v.loewis.de> parents: 37661 diff changeset	2431 if (_PyUnicode_Resize(&v, p - PyUnicode_AS_UNICODE(v)) < 0)
35485 6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2432 goto onError;
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2433 Py_XDECREF(errorHandler);
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2434 Py_XDECREF(exc);
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2435 return (PyObject *)v;
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2436
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2437 onError:
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2438 Py_XDECREF(v);
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2439 Py_XDECREF(errorHandler);
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2440 Py_XDECREF(exc);
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2441 return NULL;
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2442 }
6b8c55877755 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34016 diff changeset	2443
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2444 /* --- Latin-1 Codec ------------------------------------------------------ */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2445
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2446 PyObject PyUnicode_DecodeLatin1(const char s,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	2447 Py_ssize_t size,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2448 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2449 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2450 PyUnicodeObject *v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2451 Py_UNICODE *p;
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	2452
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2453 /* Latin-1 is equivalent to the first 256 ordinals in Unicode. */
31238 4edf200a8118 SF #859573: Reduce compiler warnings on gcc 3.2 and above. Hye-Shik Chang <hyeshik@gmail.com> parents: 31199 diff changeset	2454 if (size == 1) {
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	2455 Py_UNICODE r = (unsigned char)s;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	2456 return PyUnicode_FromUnicode(&r, 1);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	2457 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	2458
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2459 v = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2460 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2461 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2462 if (size == 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2463 return (PyObject *)v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2464 p = PyUnicode_AS_UNICODE(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2465 while (size-- > 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2466 p++ = (unsigned char)s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2467 return (PyObject *)v;
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	2468
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2469 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2470 Py_XDECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2471 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2472 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2473
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2474 /* create or adjust a UnicodeEncodeError */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2475 static void make_encode_exception(PyObject **exceptionObject,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2476 const char *encoding,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	2477 const Py_UNICODE *unicode, Py_ssize_t size,
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	2478 Py_ssize_t startpos, Py_ssize_t endpos,
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2479 const char *reason)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2480 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2481 if (*exceptionObject == NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2482 *exceptionObject = PyUnicodeEncodeError_Create(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2483 encoding, unicode, size, startpos, endpos, reason);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2484 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2485 else {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2486 if (PyUnicodeEncodeError_SetStart(*exceptionObject, startpos))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2487 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2488 if (PyUnicodeEncodeError_SetEnd(*exceptionObject, endpos))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2489 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2490 if (PyUnicodeEncodeError_SetReason(*exceptionObject, reason))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2491 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2492 return;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2493 onError:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2494 Py_DECREF(*exceptionObject);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2495 *exceptionObject = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2496 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2497 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2498
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2499 /* raises a UnicodeEncodeError */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2500 static void raise_encode_exception(PyObject **exceptionObject,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2501 const char *encoding,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	2502 const Py_UNICODE *unicode, Py_ssize_t size,
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	2503 Py_ssize_t startpos, Py_ssize_t endpos,
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2504 const char *reason)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2505 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2506 make_encode_exception(exceptionObject,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2507 encoding, unicode, size, startpos, endpos, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2508 if (*exceptionObject != NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2509 PyCodec_StrictErrors(*exceptionObject);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2510 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2511
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2512 /* error handling callback helper:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2513 build arguments, call the callback and check the arguments,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2514 put the result into newpos and return the replacement string, which
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2515 has to be freed by the caller */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2516 static PyObject unicode_encode_call_errorhandler(const char errors,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2517 PyObject **errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2518 const char encoding, const char reason,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	2519 const Py_UNICODE unicode, Py_ssize_t size, PyObject *exceptionObject,
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	2520 Py_ssize_t startpos, Py_ssize_t endpos,
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	2521 Py_ssize_t *newpos)
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	2522 {
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	2523 static char *argparse = "O!n;encoding error handler must return (unicode, int) tuple";
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2524
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2525 PyObject *restuple;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2526 PyObject *resunicode;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2527
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2528 if (*errorHandler == NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2529 *errorHandler = PyCodec_LookupError(errors);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2530 if (*errorHandler == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2531 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2532 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2533
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2534 make_encode_exception(exceptionObject,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2535 encoding, unicode, size, startpos, endpos, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2536 if (*exceptionObject == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2537 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2538
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2539 restuple = PyObject_CallFunctionObjArgs(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2540 errorHandler, exceptionObject, NULL);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2541 if (restuple == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2542 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2543 if (!PyTuple_Check(restuple)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2544 PyErr_Format(PyExc_TypeError, &argparse[4]);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2545 Py_DECREF(restuple);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2546 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2547 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2548 if (!PyArg_ParseTuple(restuple, argparse, &PyUnicode_Type,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2549 &resunicode, newpos)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2550 Py_DECREF(restuple);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2551 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2552 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2553 if (*newpos<0)
27555 c267f28724dd Change the treatment of positions returned by PEP293 Walter Dörwald <walter@livinglogic.de> parents: 27521 diff changeset	2554 newpos = size+newpos;
c267f28724dd Change the treatment of positions returned by PEP293 Walter Dörwald <walter@livinglogic.de> parents: 27521 diff changeset	2555 if (newpos<0 \|\| newpos>size) {
36510 0f3894bb2ada Support %zd in PyErr_Format and PyString_FromFormat. Martin v. Löwis <martin@v.loewis.de> parents: 36508 diff changeset	2556 PyErr_Format(PyExc_IndexError, "position %zd from error handler out of bounds", *newpos);
27555 c267f28724dd Change the treatment of positions returned by PEP293 Walter Dörwald <walter@livinglogic.de> parents: 27521 diff changeset	2557 Py_DECREF(restuple);
c267f28724dd Change the treatment of positions returned by PEP293 Walter Dörwald <walter@livinglogic.de> parents: 27521 diff changeset	2558 return NULL;
c267f28724dd Change the treatment of positions returned by PEP293 Walter Dörwald <walter@livinglogic.de> parents: 27521 diff changeset	2559 }
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2560 Py_INCREF(resunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2561 Py_DECREF(restuple);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2562 return resunicode;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2563 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2564
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2565 static PyObject unicode_encode_ucs1(const Py_UNICODE p,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	2566 Py_ssize_t size,
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2567 const char *errors,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2568 int limit)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2569 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2570 /* output object */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2571 PyObject *res;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2572 /* pointers to the beginning and end+1 of input */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2573 const Py_UNICODE *startp = p;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2574 const Py_UNICODE *endp = p + size;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2575 /* pointer to the beginning of the unencodable characters */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2576 /* const Py_UNICODE badp = NULL; /
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2577 /* pointer into the output */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2578 char *str;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2579 /* current output position */
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	2580 Py_ssize_t respos = 0;
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	2581 Py_ssize_t ressize;
37596 1b078493d538 More low-hanging fruit. Still need to re-arrange some code (or find a better Anthony Baxter <anthonybaxter@gmail.com> parents: 37306 diff changeset	2582 const char *encoding = (limit == 256) ? "latin-1" : "ascii";
1b078493d538 More low-hanging fruit. Still need to re-arrange some code (or find a better Anthony Baxter <anthonybaxter@gmail.com> parents: 37306 diff changeset	2583 const char *reason = (limit == 256) ? "ordinal not in range(256)" : "ordinal not in range(128)";
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2584 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2585 PyObject *exc = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2586 /* the following variable is used for caching string comparisons
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2587 * -1=not initialized, 0=unknown, 1=strict, 2=replace, 3=ignore, 4=xmlcharrefreplace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2588 int known_errorHandler = -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2589
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2590 /* allocate enough for a simple encoding without
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2591 replacements, if we need more, we'll resize */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2592 res = PyString_FromStringAndSize(NULL, size);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2593 if (res == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2594 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2595 if (size == 0)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2596 return res;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2597 str = PyString_AS_STRING(res);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2598 ressize = size;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2599
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2600 while (p<endp) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2601 Py_UNICODE c = *p;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2602
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2603 /* can we encode this? */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2604 if (c<limit) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2605 /* no overflow check, because we know that the space is enough */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2606 *str++ = (char)c;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2607 ++p;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2608 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2609 else {
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	2610 Py_ssize_t unicodepos = p-startp;
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	2611 Py_ssize_t requiredsize;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2612 PyObject *repunicode;
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	2613 Py_ssize_t repsize;
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	2614 Py_ssize_t newpos;
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	2615 Py_ssize_t respos;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2616 Py_UNICODE *uni2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2617 /* startpos for collecting unencodable chars */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2618 const Py_UNICODE *collstart = p;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2619 const Py_UNICODE *collend = p;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2620 /* find all unecodable characters */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2621 while ((collend < endp) && ((*collend)>=limit))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2622 ++collend;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2623 /* cache callback name lookup (if not done yet, i.e. it's the first error) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2624 if (known_errorHandler==-1) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2625 if ((errors==NULL) \|\| (!strcmp(errors, "strict")))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2626 known_errorHandler = 1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2627 else if (!strcmp(errors, "replace"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2628 known_errorHandler = 2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2629 else if (!strcmp(errors, "ignore"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2630 known_errorHandler = 3;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2631 else if (!strcmp(errors, "xmlcharrefreplace"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2632 known_errorHandler = 4;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2633 else
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2634 known_errorHandler = 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2635 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2636 switch (known_errorHandler) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2637 case 1: /* strict */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2638 raise_encode_exception(&exc, encoding, startp, size, collstart-startp, collend-startp, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2639 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2640 case 2: /* replace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2641 while (collstart++<collend)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2642 str++ = '?'; / fall through */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2643 case 3: /* ignore */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2644 p = collend;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2645 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2646 case 4: /* xmlcharrefreplace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2647 respos = str-PyString_AS_STRING(res);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2648 /* determine replacement size (temporarily (mis)uses p) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2649 for (p = collstart, repsize = 0; p < collend; ++p) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2650 if (*p<10)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2651 repsize += 2+1+1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2652 else if (*p<100)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2653 repsize += 2+2+1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2654 else if (*p<1000)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2655 repsize += 2+3+1;
31248 353a16033f6a Fix broken xmlcharrefreplace by rev 2.204. Hye-Shik Chang <hyeshik@gmail.com> parents: 31238 diff changeset	2656 else if (*p<10000)
353a16033f6a Fix broken xmlcharrefreplace by rev 2.204. Hye-Shik Chang <hyeshik@gmail.com> parents: 31238 diff changeset	2657 repsize += 2+4+1;
31238 4edf200a8118 SF #859573: Reduce compiler warnings on gcc 3.2 and above. Hye-Shik Chang <hyeshik@gmail.com> parents: 31199 diff changeset	2658 #ifndef Py_UNICODE_WIDE
4edf200a8118 SF #859573: Reduce compiler warnings on gcc 3.2 and above. Hye-Shik Chang <hyeshik@gmail.com> parents: 31199 diff changeset	2659 else
31248 353a16033f6a Fix broken xmlcharrefreplace by rev 2.204. Hye-Shik Chang <hyeshik@gmail.com> parents: 31238 diff changeset	2660 repsize += 2+5+1;
31238 4edf200a8118 SF #859573: Reduce compiler warnings on gcc 3.2 and above. Hye-Shik Chang <hyeshik@gmail.com> parents: 31199 diff changeset	2661 #else
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2662 else if (*p<100000)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2663 repsize += 2+5+1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2664 else if (*p<1000000)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2665 repsize += 2+6+1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2666 else
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2667 repsize += 2+7+1;
31238 4edf200a8118 SF #859573: Reduce compiler warnings on gcc 3.2 and above. Hye-Shik Chang <hyeshik@gmail.com> parents: 31199 diff changeset	2668 #endif
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2669 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2670 requiredsize = respos+repsize+(endp-collend);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2671 if (requiredsize > ressize) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2672 if (requiredsize<2*ressize)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2673 requiredsize = 2*ressize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2674 if (_PyString_Resize(&res, requiredsize))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2675 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2676 str = PyString_AS_STRING(res) + respos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2677 ressize = requiredsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2678 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2679 /* generate replacement (temporarily (mis)uses p) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2680 for (p = collstart; p < collend; ++p) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2681 str += sprintf(str, "&#%d;", (int)*p);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2682 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2683 p = collend;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2684 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2685 default:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2686 repunicode = unicode_encode_call_errorhandler(errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2687 encoding, reason, startp, size, &exc,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2688 collstart-startp, collend-startp, &newpos);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2689 if (repunicode == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2690 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2691 /* need more space? (at least enough for what we
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2692 have+the replacement+the rest of the string, so
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2693 we won't have to check space for encodable characters) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2694 respos = str-PyString_AS_STRING(res);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2695 repsize = PyUnicode_GET_SIZE(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2696 requiredsize = respos+repsize+(endp-collend);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2697 if (requiredsize > ressize) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2698 if (requiredsize<2*ressize)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2699 requiredsize = 2*ressize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2700 if (_PyString_Resize(&res, requiredsize)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2701 Py_DECREF(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2702 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2703 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2704 str = PyString_AS_STRING(res) + respos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2705 ressize = requiredsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2706 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2707 /* check if there is anything unencodable in the replacement
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2708 and copy it to the output */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2709 for (uni2 = PyUnicode_AS_UNICODE(repunicode);repsize-->0; ++uni2, ++str) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2710 c = *uni2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2711 if (c >= limit) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2712 raise_encode_exception(&exc, encoding, startp, size,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2713 unicodepos, unicodepos+1, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2714 Py_DECREF(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2715 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2716 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2717 *str = (char)c;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2718 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2719 p = startp + newpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2720 Py_DECREF(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2721 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2722 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2723 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2724 /* Resize if we allocated to much */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2725 respos = str-PyString_AS_STRING(res);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2726 if (respos<ressize)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2727 /* If this falls res will be NULL */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2728 _PyString_Resize(&res, respos);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2729 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2730 Py_XDECREF(exc);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2731 return res;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2732
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2733 onError:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2734 Py_XDECREF(res);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2735 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2736 Py_XDECREF(exc);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2737 return NULL;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2738 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2739
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2740 PyObject PyUnicode_EncodeLatin1(const Py_UNICODE p,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	2741 Py_ssize_t size,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2742 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2743 {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2744 return unicode_encode_ucs1(p, size, errors, 256);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2745 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2746
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2747 PyObject PyUnicode_AsLatin1String(PyObject unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2748 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2749 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2750 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2751 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2752 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2753 return PyUnicode_EncodeLatin1(PyUnicode_AS_UNICODE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2754 PyUnicode_GET_SIZE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2755 NULL);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2756 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2757
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2758 /* --- 7-bit ASCII Codec -------------------------------------------------- */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2759
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2760 PyObject PyUnicode_DecodeASCII(const char s,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	2761 Py_ssize_t size,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2762 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2763 {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2764 const char *starts = s;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2765 PyUnicodeObject *v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2766 Py_UNICODE *p;
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	2767 Py_ssize_t startinpos;
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	2768 Py_ssize_t endinpos;
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	2769 Py_ssize_t outpos;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2770 const char *e;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2771 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2772 PyObject *exc = NULL;
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	2773
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2774 /* ASCII is equivalent to the first 128 ordinals in Unicode. */
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	2775 if (size == 1 && (unsigned char)s < 128) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	2776 Py_UNICODE r = (unsigned char)s;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	2777 return PyUnicode_FromUnicode(&r, 1);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	2778 }
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	2779
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2780 v = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2781 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2782 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2783 if (size == 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2784 return (PyObject *)v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2785 p = PyUnicode_AS_UNICODE(v);
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2786 e = s + size;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2787 while (s < e) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2788 register unsigned char c = (unsigned char)*s;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2789 if (c < 128) {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2790 *p++ = c;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2791 ++s;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2792 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2793 else {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2794 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2795 endinpos = startinpos + 1;
30439 30e495efe90a Double-fix of crash in Unicode freelist handling. Jeremy Hylton <jeremy@alum.mit.edu> parents: 30431 diff changeset	2796 outpos = p - (Py_UNICODE *)PyUnicode_AS_UNICODE(v);
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2797 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2798 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2799 "ascii", "ordinal not in range(128)",
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2800 starts, size, &startinpos, &endinpos, &exc, &s,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2801 (PyObject **)&v, &outpos, &p))
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2802 goto onError;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2803 }
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2804 }
12329 3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	2805 if (p - PyUnicode_AS_UNICODE(v) < PyString_GET_SIZE(v))
37662 0678ce9fb6b2 Change more ints to Py_ssize_t. Martin v. Löwis <martin@v.loewis.de> parents: 37661 diff changeset	2806 if (_PyUnicode_Resize(&v, p - PyUnicode_AS_UNICODE(v)) < 0)
12329 3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	2807 goto onError;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2808 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2809 Py_XDECREF(exc);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2810 return (PyObject *)v;
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	2811
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2812 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2813 Py_XDECREF(v);
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2814 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2815 Py_XDECREF(exc);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2816 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2817 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2818
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2819 PyObject PyUnicode_EncodeASCII(const Py_UNICODE p,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	2820 Py_ssize_t size,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2821 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2822 {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2823 return unicode_encode_ucs1(p, size, errors, 128);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2824 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2825
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2826 PyObject PyUnicode_AsASCIIString(PyObject unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2827 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2828 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2829 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2830 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2831 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2832 return PyUnicode_EncodeASCII(PyUnicode_AS_UNICODE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2833 PyUnicode_GET_SIZE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2834 NULL);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2835 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2836
24071 8142f8d668a0 Patch #569753: Remove support for WIN16. Martin v. Löwis <martin@v.loewis.de> parents: 23837 diff changeset	2837 #if defined(MS_WINDOWS) && defined(HAVE_USABLE_WCHAR_T)
12193 c3fc2b4617fc Some blank lines. Guido van Rossum <guido@python.org> parents: 12067 diff changeset	2838
12067 bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2839 /* --- MBCS codecs for Windows -------------------------------------------- */
12193 c3fc2b4617fc Some blank lines. Guido van Rossum <guido@python.org> parents: 12067 diff changeset	2840
38817 549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2841 #if SIZEOF_INT < SIZEOF_SSIZE_T
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2842 #define NEED_RETRY
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2843 #endif
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2844
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2845 /* XXX This code is limited to "true" double-byte encodings, as
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2846 a) it assumes an incomplete character consists of a single byte, and
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2847 b) IsDBCSLeadByte (probably) does not work for non-DBCS multi-byte
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2848 encodings, see IsDBCSLeadByteEx documentation. */
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2849
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2850 static int is_dbcs_lead_byte(const char *s, int offset)
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2851 {
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2852 const char *curr = s + offset;
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2853
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2854 if (IsDBCSLeadByte(*curr)) {
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2855 const char *prev = CharPrev(s, curr);
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2856 return (prev == curr) \|\| !IsDBCSLeadByte(*prev) \|\| (curr - prev == 2);
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2857 }
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2858 return 0;
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2859 }
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2860
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2861 /*
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2862 * Decode MBCS string into unicode object. If 'final' is set, converts
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2863 * trailing lead-byte too. Returns consumed size if succeed, -1 otherwise.
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2864 */
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2865 static int decode_mbcs(PyUnicodeObject **v,
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2866 const char s, / MBCS string */
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2867 int size, /* sizeof MBCS string */
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2868 int final)
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2869 {
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2870 Py_UNICODE *p;
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2871 Py_ssize_t n = 0;
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2872 int usize = 0;
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2873
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2874 assert(size >= 0);
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2875
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2876 /* Skip trailing lead-byte unless 'final' is set */
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2877 if (!final && size >= 1 && is_dbcs_lead_byte(s, size - 1))
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2878 --size;
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2879
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2880 /* First get the size of the result */
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2881 if (size > 0) {
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2882 usize = MultiByteToWideChar(CP_ACP, 0, s, size, NULL, 0);
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2883 if (usize == 0) {
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2884 PyErr_SetFromWindowsErrWithFilename(0, NULL);
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2885 return -1;
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2886 }
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2887 }
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2888
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2889 if (*v == NULL) {
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2890 /* Create unicode object */
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2891 *v = _PyUnicode_New(usize);
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2892 if (*v == NULL)
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2893 return -1;
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2894 }
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2895 else {
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2896 /* Extend unicode object */
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2897 n = PyUnicode_GET_SIZE(*v);
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2898 if (_PyUnicode_Resize(v, n + usize) < 0)
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2899 return -1;
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2900 }
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2901
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2902 /* Do the conversion */
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2903 if (size > 0) {
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2904 p = PyUnicode_AS_UNICODE(*v) + n;
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2905 if (0 == MultiByteToWideChar(CP_ACP, 0, s, size, p, usize)) {
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2906 PyErr_SetFromWindowsErrWithFilename(0, NULL);
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2907 return -1;
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2908 }
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2909 }
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2910
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2911 return size;
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2912 }
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2913
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2914 PyObject PyUnicode_DecodeMBCSStateful(const char s,
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2915 Py_ssize_t size,
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2916 const char *errors,
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2917 Py_ssize_t *consumed)
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2918 {
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2919 PyUnicodeObject *v = NULL;
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2920 int done;
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2921
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2922 if (consumed)
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2923 *consumed = 0;
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2924
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2925 #ifdef NEED_RETRY
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2926 retry:
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2927 if (size > INT_MAX)
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2928 done = decode_mbcs(&v, s, INT_MAX, 0);
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2929 else
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2930 #endif
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2931 done = decode_mbcs(&v, s, (int)size, !consumed);
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2932
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2933 if (done < 0) {
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2934 Py_XDECREF(v);
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2935 return NULL;
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2936 }
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2937
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2938 if (consumed)
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2939 *consumed += done;
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2940
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2941 #ifdef NEED_RETRY
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2942 if (size > INT_MAX) {
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2943 s += done;
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2944 size -= done;
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2945 goto retry;
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2946 }
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2947 #endif
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2948
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2949 return (PyObject *)v;
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2950 }
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2951
12067 bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2952 PyObject PyUnicode_DecodeMBCS(const char s,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	2953 Py_ssize_t size,
12067 bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2954 const char *errors)
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2955 {
38817 549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2956 return PyUnicode_DecodeMBCSStateful(s, size, errors, NULL);
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2957 }
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2958
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2959 /*
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2960 * Convert unicode into string object (MBCS).
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2961 * Returns 0 if succeed, -1 otherwise.
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2962 */
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2963 static int encode_mbcs(PyObject **repr,
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2964 const Py_UNICODE p, / unicode */
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2965 int size) /* size of unicode */
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2966 {
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2967 int mbcssize = 0;
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2968 Py_ssize_t n = 0;
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2969
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2970 assert(size >= 0);
12067 bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2971
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2972 /* First get the size of the result */
38817 549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2973 if (size > 0) {
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2974 mbcssize = WideCharToMultiByte(CP_ACP, 0, p, size, NULL, 0, NULL, NULL);
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2975 if (mbcssize == 0) {
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2976 PyErr_SetFromWindowsErrWithFilename(0, NULL);
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2977 return -1;
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2978 }
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2979 }
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2980
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2981 if (*repr == NULL) {
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2982 /* Create string object */
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2983 *repr = PyString_FromStringAndSize(NULL, mbcssize);
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2984 if (*repr == NULL)
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2985 return -1;
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2986 }
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2987 else {
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2988 /* Extend string object */
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2989 n = PyString_Size(*repr);
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2990 if (_PyString_Resize(repr, n + mbcssize) < 0)
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2991 return -1;
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2992 }
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2993
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2994 /* Do the conversion */
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2995 if (size > 0) {
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2996 char s = PyString_AS_STRING(repr) + n;
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2997 if (0 == WideCharToMultiByte(CP_ACP, 0, p, size, s, mbcssize, NULL, NULL)) {
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2998 PyErr_SetFromWindowsErrWithFilename(0, NULL);
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	2999 return -1;
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	3000 }
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	3001 }
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	3002
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	3003 return 0;
12067 bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	3004 }
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	3005
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	3006 PyObject PyUnicode_EncodeMBCS(const Py_UNICODE p,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	3007 Py_ssize_t size,
12067 bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	3008 const char *errors)
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	3009 {
38817 549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	3010 PyObject *repr = NULL;
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	3011 int ret;
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	3012
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	3013 #ifdef NEED_RETRY
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	3014 retry:
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	3015 if (size > INT_MAX)
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	3016 ret = encode_mbcs(&repr, p, INT_MAX);
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	3017 else
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	3018 #endif
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	3019 ret = encode_mbcs(&repr, p, (int)size);
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	3020
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	3021 if (ret < 0) {
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	3022 Py_XDECREF(repr);
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	3023 return NULL;
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	3024 }
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	3025
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	3026 #ifdef NEED_RETRY
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	3027 if (size > INT_MAX) {
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	3028 p += INT_MAX;
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	3029 size -= INT_MAX;
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	3030 goto retry;
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	3031 }
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	3032 #endif
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	3033
12067 bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	3034 return repr;
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	3035 }
12193 c3fc2b4617fc Some blank lines. Guido van Rossum <guido@python.org> parents: 12067 diff changeset	3036
29706 a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without Mark Hammond <mhammond@skippinet.com.au> parents: 29641 diff changeset	3037 PyObject PyUnicode_AsMBCSString(PyObject unicode)
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without Mark Hammond <mhammond@skippinet.com.au> parents: 29641 diff changeset	3038 {
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without Mark Hammond <mhammond@skippinet.com.au> parents: 29641 diff changeset	3039 if (!PyUnicode_Check(unicode)) {
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without Mark Hammond <mhammond@skippinet.com.au> parents: 29641 diff changeset	3040 PyErr_BadArgument();
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without Mark Hammond <mhammond@skippinet.com.au> parents: 29641 diff changeset	3041 return NULL;
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without Mark Hammond <mhammond@skippinet.com.au> parents: 29641 diff changeset	3042 }
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without Mark Hammond <mhammond@skippinet.com.au> parents: 29641 diff changeset	3043 return PyUnicode_EncodeMBCS(PyUnicode_AS_UNICODE(unicode),
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without Mark Hammond <mhammond@skippinet.com.au> parents: 29641 diff changeset	3044 PyUnicode_GET_SIZE(unicode),
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without Mark Hammond <mhammond@skippinet.com.au> parents: 29641 diff changeset	3045 NULL);
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without Mark Hammond <mhammond@skippinet.com.au> parents: 29641 diff changeset	3046 }
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without Mark Hammond <mhammond@skippinet.com.au> parents: 29641 diff changeset	3047
38817 549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	3048 #undef NEED_RETRY
549c547700af Patch #1455898: Incremental mode for "mbcs" codec. Martin v. Löwis <martin@v.loewis.de> parents: 38786 diff changeset	3049
24071 8142f8d668a0 Patch #569753: Remove support for WIN16. Martin v. Löwis <martin@v.loewis.de> parents: 23837 diff changeset	3050 #endif /* MS_WINDOWS */
12067 bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	3051
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3052 /* --- Character Mapping Codec -------------------------------------------- */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3053
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3054 PyObject PyUnicode_DecodeCharmap(const char s,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	3055 Py_ssize_t size,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3056 PyObject *mapping,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3057 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3058 {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3059 const char *starts = s;
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	3060 Py_ssize_t startinpos;
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	3061 Py_ssize_t endinpos;
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	3062 Py_ssize_t outpos;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3063 const char *e;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3064 PyUnicodeObject *v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3065 Py_UNICODE *p;
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	3066 Py_ssize_t extrachars = 0;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3067 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3068 PyObject *exc = NULL;
35702 22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3069 Py_UNICODE *mapstring = NULL;
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	3070 Py_ssize_t maplen = 0;
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	3071
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3072 /* Default to Latin-1 */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3073 if (mapping == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3074 return PyUnicode_DecodeLatin1(s, size, errors);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3075
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3076 v = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3077 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3078 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3079 if (size == 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3080 return (PyObject *)v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3081 p = PyUnicode_AS_UNICODE(v);
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3082 e = s + size;
35702 22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3083 if (PyUnicode_CheckExact(mapping)) {
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3084 mapstring = PyUnicode_AS_UNICODE(mapping);
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3085 maplen = PyUnicode_GET_SIZE(mapping);
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3086 while (s < e) {
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3087 unsigned char ch = *s;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3088 Py_UNICODE x = 0xfffe; /* illegal value */
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3089
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3090 if (ch < maplen)
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3091 x = mapstring[ch];
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3092
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3093 if (x == 0xfffe) {
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3094 /* undefined mapping */
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3095 outpos = p-PyUnicode_AS_UNICODE(v);
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3096 startinpos = s-starts;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3097 endinpos = startinpos+1;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3098 if (unicode_decode_call_errorhandler(
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3099 errors, &errorHandler,
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3100 "charmap", "character maps to <undefined>",
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3101 starts, size, &startinpos, &endinpos, &exc, &s,
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3102 (PyObject **)&v, &outpos, &p)) {
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3103 goto onError;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3104 }
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3105 continue;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3106 }
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3107 *p++ = x;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3108 ++s;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3109 }
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3110 }
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3111 else {
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3112 while (s < e) {
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3113 unsigned char ch = *s;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3114 PyObject w, x;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3115
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3116 /* Get mapping (char ordinal -> integer, Unicode char or None) */
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3117 w = PyInt_FromLong((long)ch);
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3118 if (w == NULL)
15910 f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core Marc-André Lemburg <mal@egenix.com> parents: 15749 diff changeset	3119 goto onError;
35702 22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3120 x = PyObject_GetItem(mapping, w);
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3121 Py_DECREF(w);
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3122 if (x == NULL) {
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3123 if (PyErr_ExceptionMatches(PyExc_LookupError)) {
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3124 /* No mapping found means: mapping is undefined. */
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3125 PyErr_Clear();
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3126 x = Py_None;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3127 Py_INCREF(x);
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3128 } else
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3129 goto onError;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3130 }
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3131
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3132 /* Apply mapping */
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3133 if (PyInt_Check(x)) {
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3134 long value = PyInt_AS_LONG(x);
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3135 if (value < 0 \|\| value > 65535) {
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3136 PyErr_SetString(PyExc_TypeError,
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3137 "character mapping must be in range(65536)");
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3138 Py_DECREF(x);
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3139 goto onError;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3140 }
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3141 *p++ = (Py_UNICODE)value;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3142 }
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3143 else if (x == Py_None) {
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3144 /* undefined mapping */
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3145 outpos = p-PyUnicode_AS_UNICODE(v);
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3146 startinpos = s-starts;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3147 endinpos = startinpos+1;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3148 if (unicode_decode_call_errorhandler(
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3149 errors, &errorHandler,
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3150 "charmap", "character maps to <undefined>",
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3151 starts, size, &startinpos, &endinpos, &exc, &s,
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3152 (PyObject **)&v, &outpos, &p)) {
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3153 Py_DECREF(x);
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3154 goto onError;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3155 }
35962 73ebdc62bf8f Fix leaked reference to None. Walter Dörwald <walter@livinglogic.de> parents: 35827 diff changeset	3156 Py_DECREF(x);
35702 22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3157 continue;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3158 }
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3159 else if (PyUnicode_Check(x)) {
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	3160 Py_ssize_t targetsize = PyUnicode_GET_SIZE(x);
35702 22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3161
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3162 if (targetsize == 1)
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3163 /* 1-1 mapping */
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3164 p++ = PyUnicode_AS_UNICODE(x);
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3165
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3166 else if (targetsize > 1) {
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3167 /* 1-n mapping */
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3168 if (targetsize > extrachars) {
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3169 /* resize first */
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	3170 Py_ssize_t oldpos = p - PyUnicode_AS_UNICODE(v);
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	3171 Py_ssize_t needed = (targetsize - extrachars) + \
35702 22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3172 (targetsize << 2);
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3173 extrachars += needed;
39995 6ba402b60041 Forward-port of r52136,52138: a review of overflow-detecting code. Armin Rigo <arigo@tunes.org> parents: 39734 diff changeset	3174 /* XXX overflow detection missing */
35702 22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3175 if (_PyUnicode_Resize(&v,
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3176 PyUnicode_GET_SIZE(v) + needed) < 0) {
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3177 Py_DECREF(x);
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3178 goto onError;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3179 }
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3180 p = PyUnicode_AS_UNICODE(v) + oldpos;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3181 }
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3182 Py_UNICODE_COPY(p,
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3183 PyUnicode_AS_UNICODE(x),
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3184 targetsize);
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3185 p += targetsize;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3186 extrachars -= targetsize;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3187 }
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3188 /* 1-0 mapping: skip the character */
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3189 }
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3190 else {
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3191 /* wrong return value */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3192 PyErr_SetString(PyExc_TypeError,
35702 22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3193 "character mapping must return integer, None or unicode");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3194 Py_DECREF(x);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3195 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3196 }
35702 22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3197 Py_DECREF(x);
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending Walter Dörwald <walter@livinglogic.de> parents: 35485 diff changeset	3198 ++s;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3199 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3200 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3201 if (p - PyUnicode_AS_UNICODE(v) < PyUnicode_GET_SIZE(v))
37662 0678ce9fb6b2 Change more ints to Py_ssize_t. Martin v. Löwis <martin@v.loewis.de> parents: 37661 diff changeset	3202 if (_PyUnicode_Resize(&v, p - PyUnicode_AS_UNICODE(v)) < 0)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3203 goto onError;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3204 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3205 Py_XDECREF(exc);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3206 return (PyObject *)v;
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	3207
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3208 onError:
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3209 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3210 Py_XDECREF(exc);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3211 Py_XDECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3212 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3213 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3214
38586 9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3215 /* Charmap encoding: the lookup table */
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3216
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3217 struct encoding_map{
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3218 PyObject_HEAD
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3219 unsigned char level1[32];
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3220 int count2, count3;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3221 unsigned char level23[1];
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3222 };
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3223
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3224 static PyObject*
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3225 encoding_map_size(PyObject obj, PyObject args)
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3226 {
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3227 struct encoding_map map = (struct encoding_map)obj;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3228 return PyInt_FromLong(sizeof(map) - 1 + 16map->count2 +
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3229 128*map->count3);
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3230 }
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3231
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3232 static PyMethodDef encoding_map_methods[] = {
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3233 {"size", encoding_map_size, METH_NOARGS,
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3234 PyDoc_STR("Return the size (in bytes) of this object") },
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3235 { 0 }
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3236 };
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3237
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3238 static void
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3239 encoding_map_dealloc(PyObject* o)
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3240 {
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3241 PyObject_FREE(o);
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3242 }
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3243
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3244 static PyTypeObject EncodingMapType = {
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3245 PyObject_HEAD_INIT(NULL)
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3246 0, /ob_size/
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3247 "EncodingMap", /tp_name/
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3248 sizeof(struct encoding_map), /tp_basicsize/
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3249 0, /tp_itemsize/
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3250 /* methods */
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3251 encoding_map_dealloc, /tp_dealloc/
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3252 0, /tp_print/
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3253 0, /tp_getattr/
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3254 0, /tp_setattr/
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3255 0, /tp_compare/
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3256 0, /tp_repr/
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3257 0, /tp_as_number/
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3258 0, /tp_as_sequence/
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3259 0, /tp_as_mapping/
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3260 0, /tp_hash/
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3261 0, /tp_call/
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3262 0, /tp_str/
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3263 0, /tp_getattro/
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3264 0, /tp_setattro/
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3265 0, /tp_as_buffer/
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3266 Py_TPFLAGS_DEFAULT, /tp_flags/
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3267 0, /tp_doc/
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3268 0, /tp_traverse/
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3269 0, /tp_clear/
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3270 0, /tp_richcompare/
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3271 0, /tp_weaklistoffset/
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3272 0, /tp_iter/
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3273 0, /tp_iternext/
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3274 encoding_map_methods, /tp_methods/
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3275 0, /tp_members/
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3276 0, /tp_getset/
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3277 0, /tp_base/
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3278 0, /tp_dict/
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3279 0, /tp_descr_get/
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3280 0, /tp_descr_set/
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3281 0, /tp_dictoffset/
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3282 0, /tp_init/
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3283 0, /tp_alloc/
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3284 0, /tp_new/
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3285 0, /tp_free/
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3286 0, /tp_is_gc/
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3287 };
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3288
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3289 PyObject*
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3290 PyUnicode_BuildEncodingMap(PyObject* string)
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3291 {
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3292 Py_UNICODE *decode;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3293 PyObject *result;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3294 struct encoding_map *mresult;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3295 int i;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3296 int need_dict = 0;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3297 unsigned char level1[32];
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3298 unsigned char level2[512];
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3299 unsigned char mlevel1, mlevel2, *mlevel3;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3300 int count2 = 0, count3 = 0;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3301
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3302 if (!PyUnicode_Check(string) \|\| PyUnicode_GetSize(string) != 256) {
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3303 PyErr_BadArgument();
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3304 return NULL;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3305 }
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3306 decode = PyUnicode_AS_UNICODE(string);
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3307 memset(level1, 0xFF, sizeof level1);
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3308 memset(level2, 0xFF, sizeof level2);
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3309
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3310 /* If there isn't a one-to-one mapping of NULL to \0,
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3311 or if there are non-BMP characters, we need to use
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3312 a mapping dictionary. */
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3313 if (decode[0] != 0)
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3314 need_dict = 1;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3315 for (i = 1; i < 256; i++) {
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3316 int l1, l2;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3317 if (decode[i] == 0
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3318 #ifdef Py_UNICODE_WIDE
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3319 \|\| decode[i] > 0xFFFF
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3320 #endif
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3321 ) {
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3322 need_dict = 1;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3323 break;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3324 }
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3325 if (decode[i] == 0xFFFE)
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3326 /* unmapped character */
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3327 continue;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3328 l1 = decode[i] >> 11;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3329 l2 = decode[i] >> 7;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3330 if (level1[l1] == 0xFF)
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3331 level1[l1] = count2++;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3332 if (level2[l2] == 0xFF)
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3333 level2[l2] = count3++;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3334 }
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3335
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3336 if (count2 >= 0xFF \|\| count3 >= 0xFF)
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3337 need_dict = 1;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3338
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3339 if (need_dict) {
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3340 PyObject *result = PyDict_New();
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3341 PyObject key, value;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3342 if (!result)
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3343 return NULL;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3344 for (i = 0; i < 256; i++) {
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3345 key = value = NULL;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3346 key = PyInt_FromLong(decode[i]);
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3347 value = PyInt_FromLong(i);
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3348 if (!key \|\| !value)
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3349 goto failed1;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3350 if (PyDict_SetItem(result, key, value) == -1)
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3351 goto failed1;
38587 21969b71c36d Repair refleaks in unicodeobject. Georg Brandl <georg@python.org> parents: 38586 diff changeset	3352 Py_DECREF(key);
21969b71c36d Repair refleaks in unicodeobject. Georg Brandl <georg@python.org> parents: 38586 diff changeset	3353 Py_DECREF(value);
38586 9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3354 }
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3355 return result;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3356 failed1:
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3357 Py_XDECREF(key);
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3358 Py_XDECREF(value);
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3359 Py_DECREF(result);
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3360 return NULL;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3361 }
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3362
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3363 /* Create a three-level trie */
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3364 result = PyObject_MALLOC(sizeof(struct encoding_map) +
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3365 16count2 + 128count3 - 1);
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3366 if (!result)
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3367 return PyErr_NoMemory();
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3368 PyObject_Init(result, &EncodingMapType);
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3369 mresult = (struct encoding_map*)result;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3370 mresult->count2 = count2;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3371 mresult->count3 = count3;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3372 mlevel1 = mresult->level1;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3373 mlevel2 = mresult->level23;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3374 mlevel3 = mresult->level23 + 16*count2;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3375 memcpy(mlevel1, level1, 32);
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3376 memset(mlevel2, 0xFF, 16*count2);
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3377 memset(mlevel3, 0, 128*count3);
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3378 count3 = 0;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3379 for (i = 1; i < 256; i++) {
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3380 int o1, o2, o3, i2, i3;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3381 if (decode[i] == 0xFFFE)
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3382 /* unmapped character */
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3383 continue;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3384 o1 = decode[i]>>11;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3385 o2 = (decode[i]>>7) & 0xF;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3386 i2 = 16*mlevel1[o1] + o2;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3387 if (mlevel2[i2] == 0xFF)
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3388 mlevel2[i2] = count3++;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3389 o3 = decode[i] & 0x7F;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3390 i3 = 128*mlevel2[i2] + o3;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3391 mlevel3[i3] = i;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3392 }
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3393 return result;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3394 }
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3395
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3396 static int
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3397 encoding_map_lookup(Py_UNICODE c, PyObject *mapping)
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3398 {
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3399 struct encoding_map map = (struct encoding_map)mapping;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3400 int l1 = c>>11;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3401 int l2 = (c>>7) & 0xF;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3402 int l3 = c & 0x7F;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3403 int i;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3404
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3405 #ifdef Py_UNICODE_WIDE
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3406 if (c > 0xFFFF) {
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3407 return -1;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3408 }
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3409 #endif
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3410 if (c == 0)
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3411 return 0;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3412 /* level 1*/
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3413 i = map->level1[l1];
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3414 if (i == 0xFF) {
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3415 return -1;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3416 }
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3417 /* level 2*/
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3418 i = map->level23[16*i+l2];
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3419 if (i == 0xFF) {
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3420 return -1;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3421 }
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3422 /* level 3 */
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3423 i = map->level23[16map->count2 + 128i + l3];
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3424 if (i == 0) {
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3425 return -1;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3426 }
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3427 return i;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3428 }
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3429
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3430 /* Lookup the character ch in the mapping. If the character
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3431 can't be found, Py_None is returned (or NULL, if another
35806 44fe157255bc fix typos, mostly in comments Fred Drake <fdrake@acm.org> parents: 35758 diff changeset	3432 error occurred). */
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3433 static PyObject charmapencode_lookup(Py_UNICODE c, PyObject mapping)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3434 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3435 PyObject *w = PyInt_FromLong((long)c);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3436 PyObject *x;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3437
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3438 if (w == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3439 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3440 x = PyObject_GetItem(mapping, w);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3441 Py_DECREF(w);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3442 if (x == NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3443 if (PyErr_ExceptionMatches(PyExc_LookupError)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3444 /* No mapping found means: mapping is undefined. */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3445 PyErr_Clear();
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3446 x = Py_None;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3447 Py_INCREF(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3448 return x;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3449 } else
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3450 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3451 }
27175 47b134bb51c3 Fix charmapencode_lookup(), so that a None value in the mapping Walter Dörwald <walter@livinglogic.de> parents: 27170 diff changeset	3452 else if (x == Py_None)
47b134bb51c3 Fix charmapencode_lookup(), so that a None value in the mapping Walter Dörwald <walter@livinglogic.de> parents: 27170 diff changeset	3453 return x;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3454 else if (PyInt_Check(x)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3455 long value = PyInt_AS_LONG(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3456 if (value < 0 \|\| value > 255) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3457 PyErr_SetString(PyExc_TypeError,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3458 "character mapping must be in range(256)");
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3459 Py_DECREF(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3460 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3461 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3462 return x;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3463 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3464 else if (PyString_Check(x))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3465 return x;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3466 else {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3467 /* wrong return value */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3468 PyErr_SetString(PyExc_TypeError,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3469 "character mapping must return integer, None or str");
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3470 Py_DECREF(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3471 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3472 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3473 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3474
38586 9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3475 static int
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3476 charmapencode_resize(PyObject *outobj, Py_ssize_t outpos, Py_ssize_t requiredsize)
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3477 {
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3478 Py_ssize_t outsize = PyString_GET_SIZE(*outobj);
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3479 /* exponentially overallocate to minimize reallocations */
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3480 if (requiredsize < 2*outsize)
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3481 requiredsize = 2*outsize;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3482 if (_PyString_Resize(outobj, requiredsize)) {
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3483 return 0;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3484 }
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3485 return 1;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3486 }
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3487
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3488 typedef enum charmapencode_result {
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3489 enc_SUCCESS, enc_FAILED, enc_EXCEPTION
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3490 }charmapencode_result;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3491 /* lookup the character, put the result in the output string and adjust
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3492 various state variables. Reallocate the output string if not enough
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3493 space is available. Return a new reference to the object that
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3494 was put in the output buffer, or Py_None, if the mapping was undefined
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3495 (in which case no character was written) or NULL, if a
35827 ab3d1e6a0237 Another comment typo fix Andrew M. Kuchling <amk@amk.ca> parents: 35826 diff changeset	3496 reallocation error occurred. The caller must decref the result */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3497 static
38586 9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3498 charmapencode_result charmapencode_output(Py_UNICODE c, PyObject *mapping,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	3499 PyObject *outobj, Py_ssize_t outpos)
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3500 {
38586 9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3501 PyObject *rep;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3502 char *outstart;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3503 Py_ssize_t outsize = PyString_GET_SIZE(*outobj);
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3504
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3505 if (mapping->ob_type == &EncodingMapType) {
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3506 int res = encoding_map_lookup(c, mapping);
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3507 Py_ssize_t requiredsize = *outpos+1;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3508 if (res == -1)
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3509 return enc_FAILED;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3510 if (outsize<requiredsize)
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3511 if (!charmapencode_resize(outobj, outpos, requiredsize))
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3512 return enc_EXCEPTION;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3513 outstart = PyString_AS_STRING(*outobj);
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3514 outstart[(*outpos)++] = (char)res;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3515 return enc_SUCCESS;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3516 }
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3517
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3518 rep = charmapencode_lookup(c, mapping);
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3519 if (rep==NULL)
38586 9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3520 return enc_EXCEPTION;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3521 else if (rep==Py_None) {
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3522 Py_DECREF(rep);
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3523 return enc_FAILED;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3524 } else {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3525 if (PyInt_Check(rep)) {
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	3526 Py_ssize_t requiredsize = *outpos+1;
38586 9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3527 if (outsize<requiredsize)
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3528 if (!charmapencode_resize(outobj, outpos, requiredsize)) {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3529 Py_DECREF(rep);
38586 9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3530 return enc_EXCEPTION;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3531 }
38586 9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3532 outstart = PyString_AS_STRING(*outobj);
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3533 outstart[(*outpos)++] = (char)PyInt_AS_LONG(rep);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3534 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3535 else {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3536 const char *repchars = PyString_AS_STRING(rep);
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	3537 Py_ssize_t repsize = PyString_GET_SIZE(rep);
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	3538 Py_ssize_t requiredsize = *outpos+repsize;
38586 9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3539 if (outsize<requiredsize)
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3540 if (!charmapencode_resize(outobj, outpos, requiredsize)) {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3541 Py_DECREF(rep);
38586 9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3542 return enc_EXCEPTION;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3543 }
38586 9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3544 outstart = PyString_AS_STRING(*outobj);
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3545 memcpy(outstart + *outpos, repchars, repsize);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3546 *outpos += repsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3547 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3548 }
38587 21969b71c36d Repair refleaks in unicodeobject. Georg Brandl <georg@python.org> parents: 38586 diff changeset	3549 Py_DECREF(rep);
38586 9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3550 return enc_SUCCESS;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3551 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3552
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3553 /* handle an error in PyUnicode_EncodeCharmap
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3554 Return 0 on success, -1 on error */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3555 static
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3556 int charmap_encoding_error(
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	3557 const Py_UNICODE p, Py_ssize_t size, Py_ssize_t inpos, PyObject *mapping,
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3558 PyObject **exceptionObject,
30149 59bad7efff02 Fix refcount leak in PyUnicode_EncodeCharmap(). The bug surfaces Walter Dörwald <walter@livinglogic.de> parents: 29706 diff changeset	3559 int known_errorHandler, PyObject errorHandler, const char errors,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	3560 PyObject *res, Py_ssize_t respos)
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3561 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3562 PyObject repunicode = NULL; / initialize to prevent gcc warning */
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	3563 Py_ssize_t repsize;
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	3564 Py_ssize_t newpos;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3565 Py_UNICODE *uni2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3566 /* startpos for collecting unencodable chars */
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	3567 Py_ssize_t collstartpos = *inpos;
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	3568 Py_ssize_t collendpos = *inpos+1;
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	3569 Py_ssize_t collpos;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3570 char *encoding = "charmap";
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3571 char *reason = "character maps to <undefined>";
38586 9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3572 charmapencode_result x;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3573
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3574 /* find all unencodable characters */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3575 while (collendpos < size) {
38586 9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3576 PyObject *rep;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3577 if (mapping->ob_type == &EncodingMapType) {
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3578 int res = encoding_map_lookup(p[collendpos], mapping);
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3579 if (res != -1)
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3580 break;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3581 ++collendpos;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3582 continue;
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3583 }
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3584
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3585 rep = charmapencode_lookup(p[collendpos], mapping);
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3586 if (rep==NULL)
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3587 return -1;
38586 9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3588 else if (rep!=Py_None) {
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3589 Py_DECREF(rep);
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3590 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3591 }
38586 9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3592 Py_DECREF(rep);
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3593 ++collendpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3594 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3595 /* cache callback name lookup
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3596 * (if not done yet, i.e. it's the first error) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3597 if (*known_errorHandler==-1) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3598 if ((errors==NULL) \|\| (!strcmp(errors, "strict")))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3599 *known_errorHandler = 1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3600 else if (!strcmp(errors, "replace"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3601 *known_errorHandler = 2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3602 else if (!strcmp(errors, "ignore"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3603 *known_errorHandler = 3;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3604 else if (!strcmp(errors, "xmlcharrefreplace"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3605 *known_errorHandler = 4;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3606 else
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3607 *known_errorHandler = 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3608 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3609 switch (*known_errorHandler) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3610 case 1: /* strict */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3611 raise_encode_exception(exceptionObject, encoding, p, size, collstartpos, collendpos, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3612 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3613 case 2: /* replace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3614 for (collpos = collstartpos; collpos<collendpos; ++collpos) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3615 x = charmapencode_output('?', mapping, res, respos);
38586 9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3616 if (x==enc_EXCEPTION) {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3617 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3618 }
38586 9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3619 else if (x==enc_FAILED) {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3620 raise_encode_exception(exceptionObject, encoding, p, size, collstartpos, collendpos, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3621 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3622 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3623 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3624 /* fall through */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3625 case 3: /* ignore */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3626 *inpos = collendpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3627 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3628 case 4: /* xmlcharrefreplace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3629 /* generate replacement (temporarily (mis)uses p) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3630 for (collpos = collstartpos; collpos < collendpos; ++collpos) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3631 char buffer[2+29+1+1];
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3632 char *cp;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3633 sprintf(buffer, "&#%d;", (int)p[collpos]);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3634 for (cp = buffer; *cp; ++cp) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3635 x = charmapencode_output(*cp, mapping, res, respos);
38586 9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3636 if (x==enc_EXCEPTION)
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3637 return -1;
38586 9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3638 else if (x==enc_FAILED) {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3639 raise_encode_exception(exceptionObject, encoding, p, size, collstartpos, collendpos, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3640 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3641 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3642 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3643 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3644 *inpos = collendpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3645 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3646 default:
30149 59bad7efff02 Fix refcount leak in PyUnicode_EncodeCharmap(). The bug surfaces Walter Dörwald <walter@livinglogic.de> parents: 29706 diff changeset	3647 repunicode = unicode_encode_call_errorhandler(errors, errorHandler,
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3648 encoding, reason, p, size, exceptionObject,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3649 collstartpos, collendpos, &newpos);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3650 if (repunicode == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3651 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3652 /* generate replacement */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3653 repsize = PyUnicode_GET_SIZE(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3654 for (uni2 = PyUnicode_AS_UNICODE(repunicode); repsize-->0; ++uni2) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3655 x = charmapencode_output(*uni2, mapping, res, respos);
38586 9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3656 if (x==enc_EXCEPTION) {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3657 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3658 }
38586 9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3659 else if (x==enc_FAILED) {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3660 Py_DECREF(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3661 raise_encode_exception(exceptionObject, encoding, p, size, collstartpos, collendpos, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3662 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3663 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3664 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3665 *inpos = newpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3666 Py_DECREF(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3667 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3668 return 0;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3669 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3670
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3671 PyObject PyUnicode_EncodeCharmap(const Py_UNICODE p,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	3672 Py_ssize_t size,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3673 PyObject *mapping,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3674 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3675 {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3676 /* output object */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3677 PyObject *res = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3678 /* current input position */
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	3679 Py_ssize_t inpos = 0;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3680 /* current output position */
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	3681 Py_ssize_t respos = 0;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3682 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3683 PyObject *exc = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3684 /* the following variable is used for caching string comparisons
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3685 * -1=not initialized, 0=unknown, 1=strict, 2=replace,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3686 * 3=ignore, 4=xmlcharrefreplace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3687 int known_errorHandler = -1;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3688
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3689 /* Default to Latin-1 */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3690 if (mapping == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3691 return PyUnicode_EncodeLatin1(p, size, errors);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3692
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3693 /* allocate enough for a simple encoding without
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3694 replacements, if we need more, we'll resize */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3695 res = PyString_FromStringAndSize(NULL, size);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3696 if (res == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3697 goto onError;
13966 91ddeed12fa7 Fixed a couple of instances where a 0-length string was being Marc-André Lemburg <mal@egenix.com> parents: 13946 diff changeset	3698 if (size == 0)
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3699 return res;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3700
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3701 while (inpos<size) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3702 /* try to encode it */
38586 9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3703 charmapencode_result x = charmapencode_output(p[inpos], mapping, &res, &respos);
9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3704 if (x==enc_EXCEPTION) /* error */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3705 goto onError;
38586 9e0b050ce46f Patch #1359618: Speed-up charmap encoder. Martin v. Löwis <martin@v.loewis.de> parents: 38441 diff changeset	3706 if (x==enc_FAILED) { /* unencodable character */
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3707 if (charmap_encoding_error(p, size, &inpos, mapping,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3708 &exc,
30149 59bad7efff02 Fix refcount leak in PyUnicode_EncodeCharmap(). The bug surfaces Walter Dörwald <walter@livinglogic.de> parents: 29706 diff changeset	3709 &known_errorHandler, &errorHandler, errors,
30162 443533d08783 Fix another refcounting leak in PyUnicode_EncodeCharmap(). Walter Dörwald <walter@livinglogic.de> parents: 30160 diff changeset	3710 &res, &respos)) {
15910 f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core Marc-André Lemburg <mal@egenix.com> parents: 15749 diff changeset	3711 goto onError;
30162 443533d08783 Fix another refcounting leak in PyUnicode_EncodeCharmap(). Walter Dörwald <walter@livinglogic.de> parents: 30160 diff changeset	3712 }
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3713 }
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3714 else
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3715 /* done with this character => adjust input position */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3716 ++inpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3717 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3718
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3719 /* Resize if we allocated to much */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3720 if (respos<PyString_GET_SIZE(res)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3721 if (_PyString_Resize(&res, respos))
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3722 goto onError;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3723 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3724 Py_XDECREF(exc);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3725 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3726 return res;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3727
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3728 onError:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3729 Py_XDECREF(res);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3730 Py_XDECREF(exc);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3731 Py_XDECREF(errorHandler);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3732 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3733 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3734
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3735 PyObject PyUnicode_AsCharmapString(PyObject unicode,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3736 PyObject *mapping)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3737 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3738 if (!PyUnicode_Check(unicode) \|\| mapping == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3739 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3740 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3741 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3742 return PyUnicode_EncodeCharmap(PyUnicode_AS_UNICODE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3743 PyUnicode_GET_SIZE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3744 mapping,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3745 NULL);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3746 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3747
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3748 /* create or adjust a UnicodeTranslateError */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3749 static void make_translate_exception(PyObject **exceptionObject,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	3750 const Py_UNICODE *unicode, Py_ssize_t size,
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	3751 Py_ssize_t startpos, Py_ssize_t endpos,
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3752 const char *reason)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3753 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3754 if (*exceptionObject == NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3755 *exceptionObject = PyUnicodeTranslateError_Create(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3756 unicode, size, startpos, endpos, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3757 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3758 else {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3759 if (PyUnicodeTranslateError_SetStart(*exceptionObject, startpos))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3760 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3761 if (PyUnicodeTranslateError_SetEnd(*exceptionObject, endpos))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3762 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3763 if (PyUnicodeTranslateError_SetReason(*exceptionObject, reason))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3764 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3765 return;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3766 onError:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3767 Py_DECREF(*exceptionObject);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3768 *exceptionObject = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3769 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3770 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3771
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3772 /* raises a UnicodeTranslateError */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3773 static void raise_translate_exception(PyObject **exceptionObject,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	3774 const Py_UNICODE *unicode, Py_ssize_t size,
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	3775 Py_ssize_t startpos, Py_ssize_t endpos,
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3776 const char *reason)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3777 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3778 make_translate_exception(exceptionObject,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3779 unicode, size, startpos, endpos, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3780 if (*exceptionObject != NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3781 PyCodec_StrictErrors(*exceptionObject);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3782 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3783
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3784 /* error handling callback helper:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3785 build arguments, call the callback and check the arguments,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3786 put the result into newpos and return the replacement string, which
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3787 has to be freed by the caller */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3788 static PyObject unicode_translate_call_errorhandler(const char errors,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3789 PyObject **errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3790 const char *reason,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	3791 const Py_UNICODE unicode, Py_ssize_t size, PyObject *exceptionObject,
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	3792 Py_ssize_t startpos, Py_ssize_t endpos,
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	3793 Py_ssize_t *newpos)
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3794 {
37662 0678ce9fb6b2 Change more ints to Py_ssize_t. Martin v. Löwis <martin@v.loewis.de> parents: 37661 diff changeset	3795 static char *argparse = "O!n;translating error handler must return (unicode, int) tuple";
0678ce9fb6b2 Change more ints to Py_ssize_t. Martin v. Löwis <martin@v.loewis.de> parents: 37661 diff changeset	3796
0678ce9fb6b2 Change more ints to Py_ssize_t. Martin v. Löwis <martin@v.loewis.de> parents: 37661 diff changeset	3797 Py_ssize_t i_newpos;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3798 PyObject *restuple;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3799 PyObject *resunicode;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3800
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3801 if (*errorHandler == NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3802 *errorHandler = PyCodec_LookupError(errors);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3803 if (*errorHandler == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3804 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3805 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3806
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3807 make_translate_exception(exceptionObject,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3808 unicode, size, startpos, endpos, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3809 if (*exceptionObject == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3810 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3811
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3812 restuple = PyObject_CallFunctionObjArgs(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3813 errorHandler, exceptionObject, NULL);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3814 if (restuple == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3815 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3816 if (!PyTuple_Check(restuple)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3817 PyErr_Format(PyExc_TypeError, &argparse[4]);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3818 Py_DECREF(restuple);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3819 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3820 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3821 if (!PyArg_ParseTuple(restuple, argparse, &PyUnicode_Type,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	3822 &resunicode, &i_newpos)) {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3823 Py_DECREF(restuple);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3824 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3825 }
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	3826 if (i_newpos<0)
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	3827 *newpos = size+i_newpos;
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	3828 else
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	3829 *newpos = i_newpos;
27555 c267f28724dd Change the treatment of positions returned by PEP293 Walter Dörwald <walter@livinglogic.de> parents: 27521 diff changeset	3830 if (newpos<0 \|\| newpos>size) {
36510 0f3894bb2ada Support %zd in PyErr_Format and PyString_FromFormat. Martin v. Löwis <martin@v.loewis.de> parents: 36508 diff changeset	3831 PyErr_Format(PyExc_IndexError, "position %zd from error handler out of bounds", *newpos);
27555 c267f28724dd Change the treatment of positions returned by PEP293 Walter Dörwald <walter@livinglogic.de> parents: 27521 diff changeset	3832 Py_DECREF(restuple);
c267f28724dd Change the treatment of positions returned by PEP293 Walter Dörwald <walter@livinglogic.de> parents: 27521 diff changeset	3833 return NULL;
c267f28724dd Change the treatment of positions returned by PEP293 Walter Dörwald <walter@livinglogic.de> parents: 27521 diff changeset	3834 }
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3835 Py_INCREF(resunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3836 Py_DECREF(restuple);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3837 return resunicode;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3838 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3839
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3840 /* Lookup the character ch in the mapping and put the result in result,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3841 which must be decrefed by the caller.
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3842 Return 0 on success, -1 on error */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3843 static
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3844 int charmaptranslate_lookup(Py_UNICODE c, PyObject mapping, PyObject *result)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3845 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3846 PyObject *w = PyInt_FromLong((long)c);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3847 PyObject *x;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3848
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3849 if (w == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3850 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3851 x = PyObject_GetItem(mapping, w);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3852 Py_DECREF(w);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3853 if (x == NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3854 if (PyErr_ExceptionMatches(PyExc_LookupError)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3855 /* No mapping found means: use 1:1 mapping. */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3856 PyErr_Clear();
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3857 *result = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3858 return 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3859 } else
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3860 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3861 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3862 else if (x == Py_None) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3863 *result = x;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3864 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3865 }
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3866 else if (PyInt_Check(x)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3867 long value = PyInt_AS_LONG(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3868 long max = PyUnicode_GetMax();
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3869 if (value < 0 \|\| value > max) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3870 PyErr_Format(PyExc_TypeError,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3871 "character mapping must be in range(0x%lx)", max+1);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3872 Py_DECREF(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3873 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3874 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3875 *result = x;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3876 return 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3877 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3878 else if (PyUnicode_Check(x)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3879 *result = x;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3880 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3881 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3882 else {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3883 /* wrong return value */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3884 PyErr_SetString(PyExc_TypeError,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3885 "character mapping must return integer, None or unicode");
30164 7cb4ad5932e6 Fix refcounting leak in charmaptranslate_lookup() Walter Dörwald <walter@livinglogic.de> parents: 30162 diff changeset	3886 Py_DECREF(x);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3887 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3888 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3889 }
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3890 /* ensure that *outobj is at least requiredsize characters long,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3891 if not reallocate and adjust various state variables.
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3892 Return 0 on success, -1 on error */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3893 static
30765 fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap(). Walter Dörwald <walter@livinglogic.de> parents: 30707 diff changeset	3894 int charmaptranslate_makespace(PyObject outobj, Py_UNICODE outp,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	3895 Py_ssize_t requiredsize)
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	3896 {
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	3897 Py_ssize_t oldsize = PyUnicode_GET_SIZE(*outobj);
30765 fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap(). Walter Dörwald <walter@livinglogic.de> parents: 30707 diff changeset	3898 if (requiredsize > oldsize) {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3899 /* remember old output position */
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	3900 Py_ssize_t outpos = outp-PyUnicode_AS_UNICODE(outobj);
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3901 /* exponentially overallocate to minimize reallocations */
30765 fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap(). Walter Dörwald <walter@livinglogic.de> parents: 30707 diff changeset	3902 if (requiredsize < 2 * oldsize)
fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap(). Walter Dörwald <walter@livinglogic.de> parents: 30707 diff changeset	3903 requiredsize = 2 * oldsize;
30431 cd0c3f916b34 Change checks of PyUnicode_Resize() return value for clarity. Jeremy Hylton <jeremy@alum.mit.edu> parents: 30216 diff changeset	3904 if (_PyUnicode_Resize(outobj, requiredsize) < 0)
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3905 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3906 outp = PyUnicode_AS_UNICODE(outobj) + outpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3907 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3908 return 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3909 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3910 /* lookup the character, put the result in the output string and adjust
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3911 various state variables. Return a new reference to the object that
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3912 was put in the output buffer in *result, or Py_None, if the mapping was
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3913 undefined (in which case no character was written).
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3914 The called must decref result.
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3915 Return 0 on success, -1 on error. */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3916 static
30765 fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap(). Walter Dörwald <walter@livinglogic.de> parents: 30707 diff changeset	3917 int charmaptranslate_output(const Py_UNICODE startinp, const Py_UNICODE curinp,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	3918 Py_ssize_t insize, PyObject mapping, PyObject outobj, Py_UNICODE *outp,
30765 fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap(). Walter Dörwald <walter@livinglogic.de> parents: 30707 diff changeset	3919 PyObject **res)
fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap(). Walter Dörwald <walter@livinglogic.de> parents: 30707 diff changeset	3920 {
fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap(). Walter Dörwald <walter@livinglogic.de> parents: 30707 diff changeset	3921 if (charmaptranslate_lookup(*curinp, mapping, res))
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3922 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3923 if (*res==NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3924 /* not found => default to 1:1 mapping */
30765 fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap(). Walter Dörwald <walter@livinglogic.de> parents: 30707 diff changeset	3925 (outp)++ = *curinp;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3926 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3927 else if (*res==Py_None)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3928 ;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3929 else if (PyInt_Check(*res)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3930 /* no overflow check, because we know that the space is enough */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3931 (outp)++ = (Py_UNICODE)PyInt_AS_LONG(*res);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3932 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3933 else if (PyUnicode_Check(*res)) {
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	3934 Py_ssize_t repsize = PyUnicode_GET_SIZE(*res);
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3935 if (repsize==1) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3936 /* no overflow check, because we know that the space is enough */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3937 (outp)++ = PyUnicode_AS_UNICODE(res);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3938 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3939 else if (repsize!=0) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3940 /* more than one character */
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	3941 Py_ssize_t requiredsize = (outp-PyUnicode_AS_UNICODE(outobj)) +
31455 37d501b9e2d5 Fix reallocation bug in unicode.translate(): The code was comparing Walter Dörwald <walter@livinglogic.de> parents: 31325 diff changeset	3942 (insize - (curinp-startinp)) +
30765 fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap(). Walter Dörwald <walter@livinglogic.de> parents: 30707 diff changeset	3943 repsize - 1;
fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap(). Walter Dörwald <walter@livinglogic.de> parents: 30707 diff changeset	3944 if (charmaptranslate_makespace(outobj, outp, requiredsize))
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3945 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3946 memcpy(outp, PyUnicode_AS_UNICODE(res), sizeof(Py_UNICODE)*repsize);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3947 *outp += repsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3948 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3949 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3950 else
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3951 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3952 return 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3953 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3954
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3955 PyObject PyUnicode_TranslateCharmap(const Py_UNICODE p,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	3956 Py_ssize_t size,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3957 PyObject *mapping,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3958 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3959 {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3960 /* output object */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3961 PyObject *res = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3962 /* pointers to the beginning and end+1 of input */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3963 const Py_UNICODE *startp = p;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3964 const Py_UNICODE *endp = p + size;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3965 /* pointer into the output */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3966 Py_UNICODE *str;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3967 /* current output position */
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	3968 Py_ssize_t respos = 0;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3969 char *reason = "character maps to <undefined>";
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3970 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3971 PyObject *exc = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3972 /* the following variable is used for caching string comparisons
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3973 * -1=not initialized, 0=unknown, 1=strict, 2=replace,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3974 * 3=ignore, 4=xmlcharrefreplace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3975 int known_errorHandler = -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3976
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3977 if (mapping == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3978 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3979 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3980 }
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3981
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3982 /* allocate enough for a simple 1:1 translation without
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3983 replacements, if we need more, we'll resize */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3984 res = PyUnicode_FromUnicode(NULL, size);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3985 if (res == NULL)
30765 fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap(). Walter Dörwald <walter@livinglogic.de> parents: 30707 diff changeset	3986 goto onError;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3987 if (size == 0)
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3988 return res;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3989 str = PyUnicode_AS_UNICODE(res);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3990
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3991 while (p<endp) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3992 /* try to encode it */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3993 PyObject *x = NULL;
30765 fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap(). Walter Dörwald <walter@livinglogic.de> parents: 30707 diff changeset	3994 if (charmaptranslate_output(startp, p, size, mapping, &res, &str, &x)) {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3995 Py_XDECREF(x);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3996 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3997 }
27774 bfc524379307 Fix two refcounting bugs Walter Dörwald <walter@livinglogic.de> parents: 27555 diff changeset	3998 Py_XDECREF(x);
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3999 if (x!=Py_None) /* it worked => adjust input pointer */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4000 ++p;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4001 else { /* untranslatable character */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4002 PyObject repunicode = NULL; / initialize to prevent gcc warning */
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	4003 Py_ssize_t repsize;
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	4004 Py_ssize_t newpos;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4005 Py_UNICODE *uni2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4006 /* startpos for collecting untranslatable chars */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4007 const Py_UNICODE *collstart = p;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4008 const Py_UNICODE *collend = p+1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4009 const Py_UNICODE *coll;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4010
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4011 /* find all untranslatable characters */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4012 while (collend < endp) {
30765 fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap(). Walter Dörwald <walter@livinglogic.de> parents: 30707 diff changeset	4013 if (charmaptranslate_lookup(*collend, mapping, &x))
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4014 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4015 Py_XDECREF(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4016 if (x!=Py_None)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4017 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4018 ++collend;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4019 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4020 /* cache callback name lookup
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4021 * (if not done yet, i.e. it's the first error) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4022 if (known_errorHandler==-1) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4023 if ((errors==NULL) \|\| (!strcmp(errors, "strict")))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4024 known_errorHandler = 1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4025 else if (!strcmp(errors, "replace"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4026 known_errorHandler = 2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4027 else if (!strcmp(errors, "ignore"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4028 known_errorHandler = 3;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4029 else if (!strcmp(errors, "xmlcharrefreplace"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4030 known_errorHandler = 4;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4031 else
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4032 known_errorHandler = 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4033 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4034 switch (known_errorHandler) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4035 case 1: /* strict */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4036 raise_translate_exception(&exc, startp, size, collstart-startp, collend-startp, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4037 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4038 case 2: /* replace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4039 /* No need to check for space, this is a 1:1 replacement */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4040 for (coll = collstart; coll<collend; ++coll)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4041 *str++ = '?';
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4042 /* fall through */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4043 case 3: /* ignore */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4044 p = collend;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4045 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4046 case 4: /* xmlcharrefreplace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4047 /* generate replacement (temporarily (mis)uses p) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4048 for (p = collstart; p < collend; ++p) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4049 char buffer[2+29+1+1];
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4050 char *cp;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4051 sprintf(buffer, "&#%d;", (int)*p);
30765 fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap(). Walter Dörwald <walter@livinglogic.de> parents: 30707 diff changeset	4052 if (charmaptranslate_makespace(&res, &str,
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4053 (str-PyUnicode_AS_UNICODE(res))+strlen(buffer)+(endp-collend)))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4054 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4055 for (cp = buffer; *cp; ++cp)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4056 str++ = cp;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4057 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4058 p = collend;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4059 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4060 default:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4061 repunicode = unicode_translate_call_errorhandler(errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4062 reason, startp, size, &exc,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4063 collstart-startp, collend-startp, &newpos);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4064 if (repunicode == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4065 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4066 /* generate replacement */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4067 repsize = PyUnicode_GET_SIZE(repunicode);
30765 fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap(). Walter Dörwald <walter@livinglogic.de> parents: 30707 diff changeset	4068 if (charmaptranslate_makespace(&res, &str,
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4069 (str-PyUnicode_AS_UNICODE(res))+repsize+(endp-collend))) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4070 Py_DECREF(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4071 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4072 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4073 for (uni2 = PyUnicode_AS_UNICODE(repunicode); repsize-->0; ++uni2)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4074 str++ = uni2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4075 p = startp + newpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4076 Py_DECREF(repunicode);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4077 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4078 }
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4079 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4080 /* Resize if we allocated to much */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4081 respos = str-PyUnicode_AS_UNICODE(res);
30765 fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap(). Walter Dörwald <walter@livinglogic.de> parents: 30707 diff changeset	4082 if (respos<PyUnicode_GET_SIZE(res)) {
30431 cd0c3f916b34 Change checks of PyUnicode_Resize() return value for clarity. Jeremy Hylton <jeremy@alum.mit.edu> parents: 30216 diff changeset	4083 if (_PyUnicode_Resize(&res, respos) < 0)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4084 goto onError;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4085 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4086 Py_XDECREF(exc);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4087 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4088 return res;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4089
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4090 onError:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4091 Py_XDECREF(res);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4092 Py_XDECREF(exc);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4093 Py_XDECREF(errorHandler);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4094 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4095 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4096
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4097 PyObject PyUnicode_Translate(PyObject str,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4098 PyObject *mapping,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4099 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4100 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4101 PyObject *result;
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	4102
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4103 str = PyUnicode_FromObject(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4104 if (str == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4105 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4106 result = PyUnicode_TranslateCharmap(PyUnicode_AS_UNICODE(str),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4107 PyUnicode_GET_SIZE(str),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4108 mapping,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4109 errors);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4110 Py_DECREF(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4111 return result;
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	4112
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4113 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4114 Py_XDECREF(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4115 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4116 }
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	4117
12248 70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	4118 /* --- Decimal Encoder ---------------------------------------------------- */
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	4119
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	4120 int PyUnicode_EncodeDecimal(Py_UNICODE *s,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	4121 Py_ssize_t length,
12248 70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	4122 char *output,
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	4123 const char *errors)
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	4124 {
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	4125 Py_UNICODE p, end;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4126 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4127 PyObject *exc = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4128 const char *encoding = "decimal";
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4129 const char *reason = "invalid decimal Unicode string";
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4130 /* the following variable is used for caching string comparisons
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4131 * -1=not initialized, 0=unknown, 1=strict, 2=replace, 3=ignore, 4=xmlcharrefreplace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4132 int known_errorHandler = -1;
12248 70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	4133
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	4134 if (output == NULL) {
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	4135 PyErr_BadArgument();
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	4136 return -1;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	4137 }
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	4138
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	4139 p = s;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	4140 end = s + length;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	4141 while (p < end) {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4142 register Py_UNICODE ch = *p;
12248 70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	4143 int decimal;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4144 PyObject *repunicode;
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	4145 Py_ssize_t repsize;
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	4146 Py_ssize_t newpos;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4147 Py_UNICODE *uni2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4148 Py_UNICODE *collstart;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4149 Py_UNICODE *collend;
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	4150
12248 70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	4151 if (Py_UNICODE_ISSPACE(ch)) {
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	4152 *output++ = ' ';
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4153 ++p;
12248 70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	4154 continue;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	4155 }
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	4156 decimal = Py_UNICODE_TODECIMAL(ch);
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	4157 if (decimal >= 0) {
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	4158 *output++ = '0' + decimal;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4159 ++p;
12248 70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	4160 continue;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	4161 }
12275 cbfa95a6a03e Conrad Huang points out that "if (0 < ch < 256)", while legal C, Guido van Rossum <guido@python.org> parents: 12249 diff changeset	4162 if (0 < ch && ch < 256) {
12577 85f00f8cd674 Fix warning detected by VC++ on assignment of Py_UNICODE to char. Guido van Rossum <guido@python.org> parents: 12576 diff changeset	4163 *output++ = (char)ch;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4164 ++p;
12248 70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	4165 continue;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	4166 }
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4167 /* All other characters are considered unencodable */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4168 collstart = p;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4169 collend = p+1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4170 while (collend < end) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4171 if ((0 < collend && collend < 256) \|\|
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4172 !Py_UNICODE_ISSPACE(*collend) \|\|
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4173 Py_UNICODE_TODECIMAL(*collend))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4174 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4175 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4176 /* cache callback name lookup
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4177 * (if not done yet, i.e. it's the first error) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4178 if (known_errorHandler==-1) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4179 if ((errors==NULL) \|\| (!strcmp(errors, "strict")))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4180 known_errorHandler = 1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4181 else if (!strcmp(errors, "replace"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4182 known_errorHandler = 2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4183 else if (!strcmp(errors, "ignore"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4184 known_errorHandler = 3;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4185 else if (!strcmp(errors, "xmlcharrefreplace"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4186 known_errorHandler = 4;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4187 else
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4188 known_errorHandler = 0;
12248 70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	4189 }
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4190 switch (known_errorHandler) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4191 case 1: /* strict */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4192 raise_encode_exception(&exc, encoding, s, length, collstart-s, collend-s, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4193 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4194 case 2: /* replace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4195 for (p = collstart; p < collend; ++p)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4196 *output++ = '?';
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4197 /* fall through */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4198 case 3: /* ignore */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4199 p = collend;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4200 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4201 case 4: /* xmlcharrefreplace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4202 /* generate replacement (temporarily (mis)uses p) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4203 for (p = collstart; p < collend; ++p)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4204 output += sprintf(output, "&#%d;", (int)*p);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4205 p = collend;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4206 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4207 default:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4208 repunicode = unicode_encode_call_errorhandler(errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4209 encoding, reason, s, length, &exc,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4210 collstart-s, collend-s, &newpos);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4211 if (repunicode == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4212 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4213 /* generate replacement */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4214 repsize = PyUnicode_GET_SIZE(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4215 for (uni2 = PyUnicode_AS_UNICODE(repunicode); repsize-->0; ++uni2) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4216 Py_UNICODE ch = *uni2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4217 if (Py_UNICODE_ISSPACE(ch))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4218 *output++ = ' ';
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4219 else {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4220 decimal = Py_UNICODE_TODECIMAL(ch);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4221 if (decimal >= 0)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4222 *output++ = '0' + decimal;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4223 else if (0 < ch && ch < 256)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4224 *output++ = (char)ch;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4225 else {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4226 Py_DECREF(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4227 raise_encode_exception(&exc, encoding,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4228 s, length, collstart-s, collend-s, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4229 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4230 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4231 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4232 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4233 p = s + newpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4234 Py_DECREF(repunicode);
12248 70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	4235 }
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	4236 }
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	4237 /* 0-terminate the output string */
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	4238 *output++ = '\0';
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4239 Py_XDECREF(exc);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4240 Py_XDECREF(errorHandler);
12248 70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	4241 return 0;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	4242
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	4243 onError:
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4244 Py_XDECREF(exc);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4245 Py_XDECREF(errorHandler);
12248 70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	4246 return -1;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	4247 }
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	4248
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4249 /* --- Helpers ------------------------------------------------------------ */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4250
38381 b32f9bb2e077 needforspeed: stringlib refactoring (in progress) Fredrik Lundh <fredrik@pythonware.com> parents: 38347 diff changeset	4251 #define STRINGLIB_CHAR Py_UNICODE
b32f9bb2e077 needforspeed: stringlib refactoring (in progress) Fredrik Lundh <fredrik@pythonware.com> parents: 38347 diff changeset	4252
38424 3d81faf47af7 needforspeed: more stringlib refactoring Fredrik Lundh <fredrik@pythonware.com> parents: 38422 diff changeset	4253 #define STRINGLIB_LEN PyUnicode_GET_SIZE
38383 05744917b7c3 needforspeed: stringlib refactoring (in progress) Fredrik Lundh <fredrik@pythonware.com> parents: 38381 diff changeset	4254 #define STRINGLIB_NEW PyUnicode_FromUnicode
38424 3d81faf47af7 needforspeed: more stringlib refactoring Fredrik Lundh <fredrik@pythonware.com> parents: 38422 diff changeset	4255 #define STRINGLIB_STR PyUnicode_AS_UNICODE
38383 05744917b7c3 needforspeed: stringlib refactoring (in progress) Fredrik Lundh <fredrik@pythonware.com> parents: 38381 diff changeset	4256
38440 5b0753b9d16a needforspeed: replace improvements, changed to Py_LOCAL_INLINE Fredrik Lundh <fredrik@pythonware.com> parents: 38427 diff changeset	4257 Py_LOCAL_INLINE(int)
38390 76807902a2f5 needforspeed: added rpartition implementation Fredrik Lundh <fredrik@pythonware.com> parents: 38383 diff changeset	4258 STRINGLIB_CMP(const Py_UNICODE* str, const Py_UNICODE* other, Py_ssize_t len)
76807902a2f5 needforspeed: added rpartition implementation Fredrik Lundh <fredrik@pythonware.com> parents: 38383 diff changeset	4259 {
38391 78771d673b4b needspeed: rpartition documentation, tests, and a bug fixes. Fredrik Lundh <fredrik@pythonware.com> parents: 38390 diff changeset	4260 if (str[0] != other[0])
78771d673b4b needspeed: rpartition documentation, tests, and a bug fixes. Fredrik Lundh <fredrik@pythonware.com> parents: 38390 diff changeset	4261 return 1;
38390 76807902a2f5 needforspeed: added rpartition implementation Fredrik Lundh <fredrik@pythonware.com> parents: 38383 diff changeset	4262 return memcmp((void) str, (void) other, len * sizeof(Py_UNICODE));
76807902a2f5 needforspeed: added rpartition implementation Fredrik Lundh <fredrik@pythonware.com> parents: 38383 diff changeset	4263 }
76807902a2f5 needforspeed: added rpartition implementation Fredrik Lundh <fredrik@pythonware.com> parents: 38383 diff changeset	4264
38383 05744917b7c3 needforspeed: stringlib refactoring (in progress) Fredrik Lundh <fredrik@pythonware.com> parents: 38381 diff changeset	4265 #define STRINGLIB_EMPTY unicode_empty
05744917b7c3 needforspeed: stringlib refactoring (in progress) Fredrik Lundh <fredrik@pythonware.com> parents: 38381 diff changeset	4266
38381 b32f9bb2e077 needforspeed: stringlib refactoring (in progress) Fredrik Lundh <fredrik@pythonware.com> parents: 38347 diff changeset	4267 #include "stringlib/fastsearch.h"
38400 33c328896f52 needforspeed: stringlib refactoring, continued. added count and Fredrik Lundh <fredrik@pythonware.com> parents: 38391 diff changeset	4268
33c328896f52 needforspeed: stringlib refactoring, continued. added count and Fredrik Lundh <fredrik@pythonware.com> parents: 38391 diff changeset	4269 #include "stringlib/count.h"
33c328896f52 needforspeed: stringlib refactoring, continued. added count and Fredrik Lundh <fredrik@pythonware.com> parents: 38391 diff changeset	4270 #include "stringlib/find.h"
38383 05744917b7c3 needforspeed: stringlib refactoring (in progress) Fredrik Lundh <fredrik@pythonware.com> parents: 38381 diff changeset	4271 #include "stringlib/partition.h"
05744917b7c3 needforspeed: stringlib refactoring (in progress) Fredrik Lundh <fredrik@pythonware.com> parents: 38381 diff changeset	4272
38402 6b03a47eb54d needforspeed: use a macro to fix slice indexes Fredrik Lundh <fredrik@pythonware.com> parents: 38401 diff changeset	4273 /* helper macro to fixup start/end slice values */
6b03a47eb54d needforspeed: use a macro to fix slice indexes Fredrik Lundh <fredrik@pythonware.com> parents: 38401 diff changeset	4274 #define FIX_START_END(obj) \
6b03a47eb54d needforspeed: use a macro to fix slice indexes Fredrik Lundh <fredrik@pythonware.com> parents: 38401 diff changeset	4275 if (start < 0) \
6b03a47eb54d needforspeed: use a macro to fix slice indexes Fredrik Lundh <fredrik@pythonware.com> parents: 38401 diff changeset	4276 start += (obj)->length; \
6b03a47eb54d needforspeed: use a macro to fix slice indexes Fredrik Lundh <fredrik@pythonware.com> parents: 38401 diff changeset	4277 if (start < 0) \
6b03a47eb54d needforspeed: use a macro to fix slice indexes Fredrik Lundh <fredrik@pythonware.com> parents: 38401 diff changeset	4278 start = 0; \
6b03a47eb54d needforspeed: use a macro to fix slice indexes Fredrik Lundh <fredrik@pythonware.com> parents: 38401 diff changeset	4279 if (end > (obj)->length) \
6b03a47eb54d needforspeed: use a macro to fix slice indexes Fredrik Lundh <fredrik@pythonware.com> parents: 38401 diff changeset	4280 end = (obj)->length; \
6b03a47eb54d needforspeed: use a macro to fix slice indexes Fredrik Lundh <fredrik@pythonware.com> parents: 38401 diff changeset	4281 if (end < 0) \
6b03a47eb54d needforspeed: use a macro to fix slice indexes Fredrik Lundh <fredrik@pythonware.com> parents: 38401 diff changeset	4282 end += (obj)->length; \
6b03a47eb54d needforspeed: use a macro to fix slice indexes Fredrik Lundh <fredrik@pythonware.com> parents: 38401 diff changeset	4283 if (end < 0) \
6b03a47eb54d needforspeed: use a macro to fix slice indexes Fredrik Lundh <fredrik@pythonware.com> parents: 38401 diff changeset	4284 end = 0;
6b03a47eb54d needforspeed: use a macro to fix slice indexes Fredrik Lundh <fredrik@pythonware.com> parents: 38401 diff changeset	4285
38400 33c328896f52 needforspeed: stringlib refactoring, continued. added count and Fredrik Lundh <fredrik@pythonware.com> parents: 38391 diff changeset	4286 Py_ssize_t PyUnicode_Count(PyObject *str,
33c328896f52 needforspeed: stringlib refactoring, continued. added count and Fredrik Lundh <fredrik@pythonware.com> parents: 38391 diff changeset	4287 PyObject *substr,
38347 6d8615f9f1cc use Py_LOCAL also for string and unicode objects Fredrik Lundh <fredrik@pythonware.com> parents: 38339 diff changeset	4288 Py_ssize_t start,
38400 33c328896f52 needforspeed: stringlib refactoring, continued. added count and Fredrik Lundh <fredrik@pythonware.com> parents: 38391 diff changeset	4289 Py_ssize_t end)
33c328896f52 needforspeed: stringlib refactoring, continued. added count and Fredrik Lundh <fredrik@pythonware.com> parents: 38391 diff changeset	4290 {
33c328896f52 needforspeed: stringlib refactoring, continued. added count and Fredrik Lundh <fredrik@pythonware.com> parents: 38391 diff changeset	4291 Py_ssize_t result;
33c328896f52 needforspeed: stringlib refactoring, continued. added count and Fredrik Lundh <fredrik@pythonware.com> parents: 38391 diff changeset	4292 PyUnicodeObject* str_obj;
33c328896f52 needforspeed: stringlib refactoring, continued. added count and Fredrik Lundh <fredrik@pythonware.com> parents: 38391 diff changeset	4293 PyUnicodeObject* sub_obj;
33c328896f52 needforspeed: stringlib refactoring, continued. added count and Fredrik Lundh <fredrik@pythonware.com> parents: 38391 diff changeset	4294
33c328896f52 needforspeed: stringlib refactoring, continued. added count and Fredrik Lundh <fredrik@pythonware.com> parents: 38391 diff changeset	4295 str_obj = (PyUnicodeObject*) PyUnicode_FromObject(str);
33c328896f52 needforspeed: stringlib refactoring, continued. added count and Fredrik Lundh <fredrik@pythonware.com> parents: 38391 diff changeset	4296 if (!str_obj)
33c328896f52 needforspeed: stringlib refactoring, continued. added count and Fredrik Lundh <fredrik@pythonware.com> parents: 38391 diff changeset	4297 return -1;
33c328896f52 needforspeed: stringlib refactoring, continued. added count and Fredrik Lundh <fredrik@pythonware.com> parents: 38391 diff changeset	4298 sub_obj = (PyUnicodeObject*) PyUnicode_FromObject(substr);
33c328896f52 needforspeed: stringlib refactoring, continued. added count and Fredrik Lundh <fredrik@pythonware.com> parents: 38391 diff changeset	4299 if (!sub_obj) {
33c328896f52 needforspeed: stringlib refactoring, continued. added count and Fredrik Lundh <fredrik@pythonware.com> parents: 38391 diff changeset	4300 Py_DECREF(str_obj);
33c328896f52 needforspeed: stringlib refactoring, continued. added count and Fredrik Lundh <fredrik@pythonware.com> parents: 38391 diff changeset	4301 return -1;
33c328896f52 needforspeed: stringlib refactoring, continued. added count and Fredrik Lundh <fredrik@pythonware.com> parents: 38391 diff changeset	4302 }
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4303
38402 6b03a47eb54d needforspeed: use a macro to fix slice indexes Fredrik Lundh <fredrik@pythonware.com> parents: 38401 diff changeset	4304 FIX_START_END(str_obj);
15910 f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core Marc-André Lemburg <mal@egenix.com> parents: 15749 diff changeset	4305
38400 33c328896f52 needforspeed: stringlib refactoring, continued. added count and Fredrik Lundh <fredrik@pythonware.com> parents: 38391 diff changeset	4306 result = stringlib_count(
33c328896f52 needforspeed: stringlib refactoring, continued. added count and Fredrik Lundh <fredrik@pythonware.com> parents: 38391 diff changeset	4307 str_obj->str + start, end - start, sub_obj->str, sub_obj->length
38275 43b1474b512b needforspeed: use "fastsearch" for count and findstring helpers. this Fredrik Lundh <fredrik@pythonware.com> parents: 38274 diff changeset	4308 );
38335 f9332ad4287a needforspeed: partition implementation, part two. Fredrik Lundh <fredrik@pythonware.com> parents: 38304 diff changeset	4309
38400 33c328896f52 needforspeed: stringlib refactoring, continued. added count and Fredrik Lundh <fredrik@pythonware.com> parents: 38391 diff changeset	4310 Py_DECREF(sub_obj);
33c328896f52 needforspeed: stringlib refactoring, continued. added count and Fredrik Lundh <fredrik@pythonware.com> parents: 38391 diff changeset	4311 Py_DECREF(str_obj);
33c328896f52 needforspeed: stringlib refactoring, continued. added count and Fredrik Lundh <fredrik@pythonware.com> parents: 38391 diff changeset	4312
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4313 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4314 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4315
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	4316 Py_ssize_t PyUnicode_Find(PyObject *str,
38424 3d81faf47af7 needforspeed: more stringlib refactoring Fredrik Lundh <fredrik@pythonware.com> parents: 38422 diff changeset	4317 PyObject *sub,
38401 adc214fc1e1c needforspeed: stringlib refactoring: use stringlib/find for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 38400 diff changeset	4318 Py_ssize_t start,
adc214fc1e1c needforspeed: stringlib refactoring: use stringlib/find for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 38400 diff changeset	4319 Py_ssize_t end,
adc214fc1e1c needforspeed: stringlib refactoring: use stringlib/find for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 38400 diff changeset	4320 int direction)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4321 {
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	4322 Py_ssize_t result;
38424 3d81faf47af7 needforspeed: more stringlib refactoring Fredrik Lundh <fredrik@pythonware.com> parents: 38422 diff changeset	4323
3d81faf47af7 needforspeed: more stringlib refactoring Fredrik Lundh <fredrik@pythonware.com> parents: 38422 diff changeset	4324 str = PyUnicode_FromObject(str);
3d81faf47af7 needforspeed: more stringlib refactoring Fredrik Lundh <fredrik@pythonware.com> parents: 38422 diff changeset	4325 if (!str)
23561 c5e8f89fd358 Fix for bug [ 561796 ] string.find causes lazy error Marc-André Lemburg <mal@egenix.com> parents: 23542 diff changeset	4326 return -2;
38424 3d81faf47af7 needforspeed: more stringlib refactoring Fredrik Lundh <fredrik@pythonware.com> parents: 38422 diff changeset	4327 sub = PyUnicode_FromObject(sub);
3d81faf47af7 needforspeed: more stringlib refactoring Fredrik Lundh <fredrik@pythonware.com> parents: 38422 diff changeset	4328 if (!sub) {
3d81faf47af7 needforspeed: more stringlib refactoring Fredrik Lundh <fredrik@pythonware.com> parents: 38422 diff changeset	4329 Py_DECREF(str);
23561 c5e8f89fd358 Fix for bug [ 561796 ] string.find causes lazy error Marc-André Lemburg <mal@egenix.com> parents: 23542 diff changeset	4330 return -2;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4331 }
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	4332
38401 adc214fc1e1c needforspeed: stringlib refactoring: use stringlib/find for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 38400 diff changeset	4333 if (direction > 0)
38441 72327f8f4a79 needforspeed: stringlib refactoring: changed find_obj to find_slice, Fredrik Lundh <fredrik@pythonware.com> parents: 38440 diff changeset	4334 result = stringlib_find_slice(
72327f8f4a79 needforspeed: stringlib refactoring: changed find_obj to find_slice, Fredrik Lundh <fredrik@pythonware.com> parents: 38440 diff changeset	4335 PyUnicode_AS_UNICODE(str), PyUnicode_GET_SIZE(str),
72327f8f4a79 needforspeed: stringlib refactoring: changed find_obj to find_slice, Fredrik Lundh <fredrik@pythonware.com> parents: 38440 diff changeset	4336 PyUnicode_AS_UNICODE(sub), PyUnicode_GET_SIZE(sub),
72327f8f4a79 needforspeed: stringlib refactoring: changed find_obj to find_slice, Fredrik Lundh <fredrik@pythonware.com> parents: 38440 diff changeset	4337 start, end
72327f8f4a79 needforspeed: stringlib refactoring: changed find_obj to find_slice, Fredrik Lundh <fredrik@pythonware.com> parents: 38440 diff changeset	4338 );
38401 adc214fc1e1c needforspeed: stringlib refactoring: use stringlib/find for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 38400 diff changeset	4339 else
38441 72327f8f4a79 needforspeed: stringlib refactoring: changed find_obj to find_slice, Fredrik Lundh <fredrik@pythonware.com> parents: 38440 diff changeset	4340 result = stringlib_rfind_slice(
72327f8f4a79 needforspeed: stringlib refactoring: changed find_obj to find_slice, Fredrik Lundh <fredrik@pythonware.com> parents: 38440 diff changeset	4341 PyUnicode_AS_UNICODE(str), PyUnicode_GET_SIZE(str),
72327f8f4a79 needforspeed: stringlib refactoring: changed find_obj to find_slice, Fredrik Lundh <fredrik@pythonware.com> parents: 38440 diff changeset	4342 PyUnicode_AS_UNICODE(sub), PyUnicode_GET_SIZE(sub),
72327f8f4a79 needforspeed: stringlib refactoring: changed find_obj to find_slice, Fredrik Lundh <fredrik@pythonware.com> parents: 38440 diff changeset	4343 start, end
72327f8f4a79 needforspeed: stringlib refactoring: changed find_obj to find_slice, Fredrik Lundh <fredrik@pythonware.com> parents: 38440 diff changeset	4344 );
38424 3d81faf47af7 needforspeed: more stringlib refactoring Fredrik Lundh <fredrik@pythonware.com> parents: 38422 diff changeset	4345
3d81faf47af7 needforspeed: more stringlib refactoring Fredrik Lundh <fredrik@pythonware.com> parents: 38422 diff changeset	4346 Py_DECREF(str);
3d81faf47af7 needforspeed: more stringlib refactoring Fredrik Lundh <fredrik@pythonware.com> parents: 38422 diff changeset	4347 Py_DECREF(sub);
3d81faf47af7 needforspeed: more stringlib refactoring Fredrik Lundh <fredrik@pythonware.com> parents: 38422 diff changeset	4348
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4349 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4350 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4351
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	4352 static
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4353 int tailmatch(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4354 PyUnicodeObject *substring,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	4355 Py_ssize_t start,
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	4356 Py_ssize_t end,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4357 int direction)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4358 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4359 if (substring->length == 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4360 return 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4361
38402 6b03a47eb54d needforspeed: use a macro to fix slice indexes Fredrik Lundh <fredrik@pythonware.com> parents: 38401 diff changeset	4362 FIX_START_END(self);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4363
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4364 end -= substring->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4365 if (end < start)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4366 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4367
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4368 if (direction > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4369 if (Py_UNICODE_MATCH(self, end, substring))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4370 return 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4371 } else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4372 if (Py_UNICODE_MATCH(self, start, substring))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4373 return 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4374 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4375
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4376 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4377 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4378
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	4379 Py_ssize_t PyUnicode_Tailmatch(PyObject *str,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4380 PyObject *substr,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	4381 Py_ssize_t start,
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	4382 Py_ssize_t end,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4383 int direction)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4384 {
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	4385 Py_ssize_t result;
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	4386
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4387 str = PyUnicode_FromObject(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4388 if (str == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4389 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4390 substr = PyUnicode_FromObject(substr);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4391 if (substr == NULL) {
36877 e9a7a2784e17 SF #1444030: Fix several potential defects found by Coverity. Hye-Shik Chang <hyeshik@gmail.com> parents: 36675 diff changeset	4392 Py_DECREF(str);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4393 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4394 }
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	4395
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4396 result = tailmatch((PyUnicodeObject *)str,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4397 (PyUnicodeObject *)substr,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4398 start, end, direction);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4399 Py_DECREF(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4400 Py_DECREF(substr);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4401 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4402 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4403
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4404 /* Apply fixfct filter to the Unicode object self and return a
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4405 reference to the modified object */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4406
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	4407 static
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4408 PyObject fixup(PyUnicodeObject self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4409 int (fixfct)(PyUnicodeObject s))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4410 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4411
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4412 PyUnicodeObject *u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4413
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	4414 u = (PyUnicodeObject*) PyUnicode_FromUnicode(NULL, self->length);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4415 if (u == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4416 return NULL;
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	4417
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	4418 Py_UNICODE_COPY(u->str, self->str, self->length);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	4419
19885 72e6eb653312 More on bug 460020: disable many optimizations of unicode subclasses. Tim Peters <tim.peters@gmail.com> parents: 19849 diff changeset	4420 if (!fixfct(u) && PyUnicode_CheckExact(self)) {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4421 /* fixfct should return TRUE if it modified the buffer. If
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4422 FALSE, return a reference to the original buffer instead
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4423 (to save space, not time) */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4424 Py_INCREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4425 Py_DECREF(u);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4426 return (PyObject*) self;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4427 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4428 return (PyObject*) u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4429 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4430
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	4431 static
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4432 int fixupper(PyUnicodeObject *self)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4433 {
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	4434 Py_ssize_t len = self->length;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4435 Py_UNICODE *s = self->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4436 int status = 0;
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	4437
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4438 while (len-- > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4439 register Py_UNICODE ch;
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	4440
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4441 ch = Py_UNICODE_TOUPPER(*s);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4442 if (ch != *s) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4443 status = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4444 *s = ch;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4445 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4446 s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4447 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4448
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4449 return status;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4450 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4451
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	4452 static
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4453 int fixlower(PyUnicodeObject *self)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4454 {
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	4455 Py_ssize_t len = self->length;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4456 Py_UNICODE *s = self->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4457 int status = 0;
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	4458
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4459 while (len-- > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4460 register Py_UNICODE ch;
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	4461
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4462 ch = Py_UNICODE_TOLOWER(*s);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4463 if (ch != *s) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4464 status = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4465 *s = ch;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4466 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4467 s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4468 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4469
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4470 return status;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4471 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4472
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	4473 static
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4474 int fixswapcase(PyUnicodeObject *self)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4475 {
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	4476 Py_ssize_t len = self->length;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4477 Py_UNICODE *s = self->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4478 int status = 0;
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	4479
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4480 while (len-- > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4481 if (Py_UNICODE_ISUPPER(*s)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4482 s = Py_UNICODE_TOLOWER(s);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4483 status = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4484 } else if (Py_UNICODE_ISLOWER(*s)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4485 s = Py_UNICODE_TOUPPER(s);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4486 status = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4487 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4488 s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4489 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4490
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4491 return status;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4492 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4493
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	4494 static
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4495 int fixcapitalize(PyUnicodeObject *self)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4496 {
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	4497 Py_ssize_t len = self->length;
16332 bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the Marc-André Lemburg <mal@egenix.com> parents: 16226 diff changeset	4498 Py_UNICODE *s = self->str;
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the Marc-André Lemburg <mal@egenix.com> parents: 16226 diff changeset	4499 int status = 0;
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	4500
16332 bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the Marc-André Lemburg <mal@egenix.com> parents: 16226 diff changeset	4501 if (len == 0)
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the Marc-André Lemburg <mal@egenix.com> parents: 16226 diff changeset	4502 return 0;
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the Marc-André Lemburg <mal@egenix.com> parents: 16226 diff changeset	4503 if (Py_UNICODE_ISLOWER(*s)) {
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the Marc-André Lemburg <mal@egenix.com> parents: 16226 diff changeset	4504 s = Py_UNICODE_TOUPPER(s);
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the Marc-André Lemburg <mal@egenix.com> parents: 16226 diff changeset	4505 status = 1;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4506 }
16332 bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the Marc-André Lemburg <mal@egenix.com> parents: 16226 diff changeset	4507 s++;
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the Marc-André Lemburg <mal@egenix.com> parents: 16226 diff changeset	4508 while (--len > 0) {
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the Marc-André Lemburg <mal@egenix.com> parents: 16226 diff changeset	4509 if (Py_UNICODE_ISUPPER(*s)) {
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the Marc-André Lemburg <mal@egenix.com> parents: 16226 diff changeset	4510 s = Py_UNICODE_TOLOWER(s);
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the Marc-André Lemburg <mal@egenix.com> parents: 16226 diff changeset	4511 status = 1;
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the Marc-André Lemburg <mal@egenix.com> parents: 16226 diff changeset	4512 }
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the Marc-André Lemburg <mal@egenix.com> parents: 16226 diff changeset	4513 s++;
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the Marc-André Lemburg <mal@egenix.com> parents: 16226 diff changeset	4514 }
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the Marc-André Lemburg <mal@egenix.com> parents: 16226 diff changeset	4515 return status;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4516 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4517
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4518 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4519 int fixtitle(PyUnicodeObject *self)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4520 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4521 register Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4522 register Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4523 int previous_is_cased;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4524
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4525 /* Shortcut for single character strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4526 if (PyUnicode_GET_SIZE(self) == 1) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4527 Py_UNICODE ch = Py_UNICODE_TOTITLE(*p);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4528 if (*p != ch) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4529 *p = ch;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4530 return 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4531 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4532 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4533 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4534 }
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	4535
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4536 e = p + PyUnicode_GET_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4537 previous_is_cased = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4538 for (; p < e; p++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4539 register const Py_UNICODE ch = *p;
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	4540
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4541 if (previous_is_cased)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4542 *p = Py_UNICODE_TOLOWER(ch);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4543 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4544 *p = Py_UNICODE_TOTITLE(ch);
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	4545
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	4546 if (Py_UNICODE_ISLOWER(ch) \|\|
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	4547 Py_UNICODE_ISUPPER(ch) \|\|
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4548 Py_UNICODE_ISTITLE(ch))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4549 previous_is_cased = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4550 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4551 previous_is_cased = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4552 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4553 return 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4554 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4555
33300 77de38534d26 PyUnicode_Join(): Two primary aims: Tim Peters <tim.peters@gmail.com> parents: 32951 diff changeset	4556 PyObject *
77de38534d26 PyUnicode_Join(): Two primary aims: Tim Peters <tim.peters@gmail.com> parents: 32951 diff changeset	4557 PyUnicode_Join(PyObject separator, PyObject seq)
77de38534d26 PyUnicode_Join(): Two primary aims: Tim Peters <tim.peters@gmail.com> parents: 32951 diff changeset	4558 {
77de38534d26 PyUnicode_Join(): Two primary aims: Tim Peters <tim.peters@gmail.com> parents: 32951 diff changeset	4559 PyObject *internal_separator = NULL;
33499 a4d3673bcd66 Initialize sep and seplen to suppress warning from gcc. Skip Montanaro <skip@pobox.com> parents: 33495 diff changeset	4560 const Py_UNICODE blank = ' ';
a4d3673bcd66 Initialize sep and seplen to suppress warning from gcc. Skip Montanaro <skip@pobox.com> parents: 33495 diff changeset	4561 const Py_UNICODE *sep = &blank;
38240 0373a074004c PyUnicode_Join(): Recent code changes introduced new Tim Peters <tim.peters@gmail.com> parents: 38239 diff changeset	4562 Py_ssize_t seplen = 1;
33310 8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4563 PyUnicodeObject res = NULL; / the result */
38240 0373a074004c PyUnicode_Join(): Recent code changes introduced new Tim Peters <tim.peters@gmail.com> parents: 38239 diff changeset	4564 Py_ssize_t res_alloc = 100; /* # allocated bytes for string in res */
0373a074004c PyUnicode_Join(): Recent code changes introduced new Tim Peters <tim.peters@gmail.com> parents: 38239 diff changeset	4565 Py_ssize_t res_used; /* # used bytes */
33310 8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4566 Py_UNICODE res_p; / pointer to free byte in res's string area */
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4567 PyObject fseq; / PySequence_Fast(seq) */
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	4568 Py_ssize_t seqlen; /* len(fseq) -- number of items in sequence */
33300 77de38534d26 PyUnicode_Join(): Two primary aims: Tim Peters <tim.peters@gmail.com> parents: 32951 diff changeset	4569 PyObject *item;
37662 0678ce9fb6b2 Change more ints to Py_ssize_t. Martin v. Löwis <martin@v.loewis.de> parents: 37661 diff changeset	4570 Py_ssize_t i;
33310 8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4571
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4572 fseq = PySequence_Fast(seq, "");
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4573 if (fseq == NULL) {
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4574 return NULL;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4575 }
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4576
33311 59bd121eeeb7 PyUnicode_Join(): Bozo Alert. While this is chugging along, it may Tim Peters <tim.peters@gmail.com> parents: 33310 diff changeset	4577 /* Grrrr. A codec may be invoked to convert str objects to
59bd121eeeb7 PyUnicode_Join(): Bozo Alert. While this is chugging along, it may Tim Peters <tim.peters@gmail.com> parents: 33310 diff changeset	4578 * Unicode, and so it's possible to call back into Python code
59bd121eeeb7 PyUnicode_Join(): Bozo Alert. While this is chugging along, it may Tim Peters <tim.peters@gmail.com> parents: 33310 diff changeset	4579 * during PyUnicode_FromObject(), and so it's possible for a sick
59bd121eeeb7 PyUnicode_Join(): Bozo Alert. While this is chugging along, it may Tim Peters <tim.peters@gmail.com> parents: 33310 diff changeset	4580 * codec to change the size of fseq (if seq is a list). Therefore
59bd121eeeb7 PyUnicode_Join(): Bozo Alert. While this is chugging along, it may Tim Peters <tim.peters@gmail.com> parents: 33310 diff changeset	4581 * we have to keep refetching the size -- can't assume seqlen
59bd121eeeb7 PyUnicode_Join(): Bozo Alert. While this is chugging along, it may Tim Peters <tim.peters@gmail.com> parents: 33310 diff changeset	4582 * is invariant.
59bd121eeeb7 PyUnicode_Join(): Bozo Alert. While this is chugging along, it may Tim Peters <tim.peters@gmail.com> parents: 33310 diff changeset	4583 */
33310 8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4584 seqlen = PySequence_Fast_GET_SIZE(fseq);
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4585 /* If empty sequence, return u"". */
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4586 if (seqlen == 0) {
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4587 res = _PyUnicode_New(0); /* empty sequence; return u"" */
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4588 goto Done;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4589 }
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4590 /* If singleton sequence with an exact Unicode, return that. */
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4591 if (seqlen == 1) {
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4592 item = PySequence_Fast_GET_ITEM(fseq, 0);
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4593 if (PyUnicode_CheckExact(item)) {
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4594 Py_INCREF(item);
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4595 res = (PyUnicodeObject *)item;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4596 goto Done;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4597 }
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4598 }
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4599
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4600 /* At least two items to join, or one that isn't exact Unicode. */
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4601 if (seqlen > 1) {
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4602 /* Set up sep and seplen -- they're needed. */
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4603 if (separator == NULL) {
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4604 sep = &blank;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4605 seplen = 1;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4606 }
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4607 else {
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4608 internal_separator = PyUnicode_FromObject(separator);
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4609 if (internal_separator == NULL)
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4610 goto onError;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4611 sep = PyUnicode_AS_UNICODE(internal_separator);
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4612 seplen = PyUnicode_GET_SIZE(internal_separator);
33311 59bd121eeeb7 PyUnicode_Join(): Bozo Alert. While this is chugging along, it may Tim Peters <tim.peters@gmail.com> parents: 33310 diff changeset	4613 /* In case PyUnicode_FromObject() mutated seq. */
59bd121eeeb7 PyUnicode_Join(): Bozo Alert. While this is chugging along, it may Tim Peters <tim.peters@gmail.com> parents: 33310 diff changeset	4614 seqlen = PySequence_Fast_GET_SIZE(fseq);
33310 8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4615 }
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4616 }
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4617
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4618 /* Get space. */
37662 0678ce9fb6b2 Change more ints to Py_ssize_t. Martin v. Löwis <martin@v.loewis.de> parents: 37661 diff changeset	4619 res = _PyUnicode_New(res_alloc);
33310 8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4620 if (res == NULL)
33300 77de38534d26 PyUnicode_Join(): Two primary aims: Tim Peters <tim.peters@gmail.com> parents: 32951 diff changeset	4621 goto onError;
33310 8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4622 res_p = PyUnicode_AS_UNICODE(res);
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4623 res_used = 0;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4624
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4625 for (i = 0; i < seqlen; ++i) {
38240 0373a074004c PyUnicode_Join(): Recent code changes introduced new Tim Peters <tim.peters@gmail.com> parents: 38239 diff changeset	4626 Py_ssize_t itemlen;
0373a074004c PyUnicode_Join(): Recent code changes introduced new Tim Peters <tim.peters@gmail.com> parents: 38239 diff changeset	4627 Py_ssize_t new_res_used;
33310 8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4628
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4629 item = PySequence_Fast_GET_ITEM(fseq, i);
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4630 /* Convert item to Unicode. */
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4631 if (! PyUnicode_Check(item) && ! PyString_Check(item)) {
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4632 PyErr_Format(PyExc_TypeError,
37775 d8b23ba4a274 Use %zd instead of %i as format character (in call to PyErr_Format) for Thomas Wouters <thomas@python.org> parents: 37702 diff changeset	4633 "sequence item %zd: expected string or Unicode,"
33310 8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4634 " %.80s found",
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4635 i, item->ob_type->tp_name);
17687 9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change Tim Peters <tim.peters@gmail.com> parents: 17640 diff changeset	4636 goto onError;
9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change Tim Peters <tim.peters@gmail.com> parents: 17640 diff changeset	4637 }
33310 8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4638 item = PyUnicode_FromObject(item);
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4639 if (item == NULL)
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4640 goto onError;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4641 /* We own a reference to item from here on. */
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4642
33311 59bd121eeeb7 PyUnicode_Join(): Bozo Alert. While this is chugging along, it may Tim Peters <tim.peters@gmail.com> parents: 33310 diff changeset	4643 /* In case PyUnicode_FromObject() mutated seq. */
59bd121eeeb7 PyUnicode_Join(): Bozo Alert. While this is chugging along, it may Tim Peters <tim.peters@gmail.com> parents: 33310 diff changeset	4644 seqlen = PySequence_Fast_GET_SIZE(fseq);
59bd121eeeb7 PyUnicode_Join(): Bozo Alert. While this is chugging along, it may Tim Peters <tim.peters@gmail.com> parents: 33310 diff changeset	4645
33310 8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4646 /* Make sure we have enough space for the separator and the item. */
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4647 itemlen = PyUnicode_GET_SIZE(item);
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4648 new_res_used = res_used + itemlen;
38709 67450db26f81 Apply perky's fix for #1503157: "/".join([u"", u""]) raising OverflowError. Georg Brandl <georg@python.org> parents: 38697 diff changeset	4649 if (new_res_used < 0)
33310 8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4650 goto Overflow;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4651 if (i < seqlen - 1) {
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4652 new_res_used += seplen;
38709 67450db26f81 Apply perky's fix for #1503157: "/".join([u"", u""]) raising OverflowError. Georg Brandl <georg@python.org> parents: 38697 diff changeset	4653 if (new_res_used < 0)
33310 8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4654 goto Overflow;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4655 }
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4656 if (new_res_used > res_alloc) {
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4657 /* double allocated size until it's big enough */
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4658 do {
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4659 res_alloc += res_alloc;
38240 0373a074004c PyUnicode_Join(): Recent code changes introduced new Tim Peters <tim.peters@gmail.com> parents: 38239 diff changeset	4660 if (res_alloc <= 0)
33310 8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4661 goto Overflow;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4662 } while (new_res_used > res_alloc);
37662 0678ce9fb6b2 Change more ints to Py_ssize_t. Martin v. Löwis <martin@v.loewis.de> parents: 37661 diff changeset	4663 if (_PyUnicode_Resize(&res, res_alloc) < 0) {
20007 56ffee1eeeec Fix Unicode .join() method to raise a TypeError for sequence Marc-André Lemburg <mal@egenix.com> parents: 19999 diff changeset	4664 Py_DECREF(item);
56ffee1eeeec Fix Unicode .join() method to raise a TypeError for sequence Marc-André Lemburg <mal@egenix.com> parents: 19999 diff changeset	4665 goto onError;
56ffee1eeeec Fix Unicode .join() method to raise a TypeError for sequence Marc-André Lemburg <mal@egenix.com> parents: 19999 diff changeset	4666 }
33310 8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4667 res_p = PyUnicode_AS_UNICODE(res) + res_used;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4668 }
33310 8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4669
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4670 /* Copy item, and maybe the separator. */
37662 0678ce9fb6b2 Change more ints to Py_ssize_t. Martin v. Löwis <martin@v.loewis.de> parents: 37661 diff changeset	4671 Py_UNICODE_COPY(res_p, PyUnicode_AS_UNICODE(item), itemlen);
33310 8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4672 res_p += itemlen;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4673 if (i < seqlen - 1) {
37662 0678ce9fb6b2 Change more ints to Py_ssize_t. Martin v. Löwis <martin@v.loewis.de> parents: 37661 diff changeset	4674 Py_UNICODE_COPY(res_p, sep, seplen);
33310 8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4675 res_p += seplen;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4676 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4677 Py_DECREF(item);
33310 8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4678 res_used = new_res_used;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4679 }
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4680
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4681 /* Shrink res to match the used area; this probably can't fail,
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4682 * but it's cheap to check.
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4683 */
37662 0678ce9fb6b2 Change more ints to Py_ssize_t. Martin v. Löwis <martin@v.loewis.de> parents: 37661 diff changeset	4684 if (_PyUnicode_Resize(&res, res_used) < 0)
33300 77de38534d26 PyUnicode_Join(): Two primary aims: Tim Peters <tim.peters@gmail.com> parents: 32951 diff changeset	4685 goto onError;
77de38534d26 PyUnicode_Join(): Two primary aims: Tim Peters <tim.peters@gmail.com> parents: 32951 diff changeset	4686
77de38534d26 PyUnicode_Join(): Two primary aims: Tim Peters <tim.peters@gmail.com> parents: 32951 diff changeset	4687 Done:
77de38534d26 PyUnicode_Join(): Two primary aims: Tim Peters <tim.peters@gmail.com> parents: 32951 diff changeset	4688 Py_XDECREF(internal_separator);
33310 8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4689 Py_DECREF(fseq);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4690 return (PyObject *)res;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4691
33300 77de38534d26 PyUnicode_Join(): Two primary aims: Tim Peters <tim.peters@gmail.com> parents: 32951 diff changeset	4692 Overflow:
77de38534d26 PyUnicode_Join(): Two primary aims: Tim Peters <tim.peters@gmail.com> parents: 32951 diff changeset	4693 PyErr_SetString(PyExc_OverflowError,
38709 67450db26f81 Apply perky's fix for #1503157: "/".join([u"", u""]) raising OverflowError. Georg Brandl <georg@python.org> parents: 38697 diff changeset	4694 "join() result is too long for a Python string");
33300 77de38534d26 PyUnicode_Join(): Two primary aims: Tim Peters <tim.peters@gmail.com> parents: 32951 diff changeset	4695 Py_DECREF(item);
77de38534d26 PyUnicode_Join(): Two primary aims: Tim Peters <tim.peters@gmail.com> parents: 32951 diff changeset	4696 /* fall through */
77de38534d26 PyUnicode_Join(): Two primary aims: Tim Peters <tim.peters@gmail.com> parents: 32951 diff changeset	4697
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4698 onError:
33300 77de38534d26 PyUnicode_Join(): Two primary aims: Tim Peters <tim.peters@gmail.com> parents: 32951 diff changeset	4699 Py_XDECREF(internal_separator);
33310 8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do Tim Peters <tim.peters@gmail.com> parents: 33304 diff changeset	4700 Py_DECREF(fseq);
17687 9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change Tim Peters <tim.peters@gmail.com> parents: 17640 diff changeset	4701 Py_XDECREF(res);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4702 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4703 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4704
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	4705 static
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	4706 PyUnicodeObject pad(PyUnicodeObject self,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	4707 Py_ssize_t left,
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	4708 Py_ssize_t right,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4709 Py_UNICODE fill)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4710 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4711 PyUnicodeObject *u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4712
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4713 if (left < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4714 left = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4715 if (right < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4716 right = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4717
19885 72e6eb653312 More on bug 460020: disable many optimizations of unicode subclasses. Tim Peters <tim.peters@gmail.com> parents: 19849 diff changeset	4718 if (left == 0 && right == 0 && PyUnicode_CheckExact(self)) {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4719 Py_INCREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4720 return self;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4721 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4722
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4723 u = _PyUnicode_New(left + self->length + right);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4724 if (u) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4725 if (left)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4726 Py_UNICODE_FILL(u->str, fill, left);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4727 Py_UNICODE_COPY(u->str + left, self->str, self->length);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4728 if (right)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4729 Py_UNICODE_FILL(u->str + left + self->length, fill, right);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4730 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4731
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4732 return u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4733 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4734
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4735 #define SPLIT_APPEND(data, left, right) \
31199 504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4736 str = PyUnicode_FromUnicode((data) + (left), (right) - (left)); \
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4737 if (!str) \
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4738 goto onError; \
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4739 if (PyList_Append(list, str)) { \
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4740 Py_DECREF(str); \
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4741 goto onError; \
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4742 } \
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4743 else \
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4744 Py_DECREF(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4745
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4746 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4747 PyObject split_whitespace(PyUnicodeObject self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4748 PyObject *list,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	4749 Py_ssize_t maxcount)
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	4750 {
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	4751 register Py_ssize_t i;
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	4752 register Py_ssize_t j;
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	4753 Py_ssize_t len = self->length;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4754 PyObject *str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4755
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4756 for (i = j = 0; i < len; ) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4757 /* find a token */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4758 while (i < len && Py_UNICODE_ISSPACE(self->str[i]))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4759 i++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4760 j = i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4761 while (i < len && !Py_UNICODE_ISSPACE(self->str[i]))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4762 i++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4763 if (j < i) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4764 if (maxcount-- <= 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4765 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4766 SPLIT_APPEND(self->str, j, i);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4767 while (i < len && Py_UNICODE_ISSPACE(self->str[i]))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4768 i++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4769 j = i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4770 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4771 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4772 if (j < len) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4773 SPLIT_APPEND(self->str, j, len);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4774 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4775 return list;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4776
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4777 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4778 Py_DECREF(list);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4779 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4780 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4781
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4782 PyObject PyUnicode_Splitlines(PyObject string,
12369 68e48bb94071 Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12329 diff changeset	4783 int keepends)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4784 {
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	4785 register Py_ssize_t i;
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	4786 register Py_ssize_t j;
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	4787 Py_ssize_t len;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4788 PyObject *list;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4789 PyObject *str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4790 Py_UNICODE *data;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4791
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4792 string = PyUnicode_FromObject(string);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4793 if (string == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4794 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4795 data = PyUnicode_AS_UNICODE(string);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4796 len = PyUnicode_GET_SIZE(string);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4797
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4798 list = PyList_New(0);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4799 if (!list)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4800 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4801
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4802 for (i = j = 0; i < len; ) {
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	4803 Py_ssize_t eol;
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	4804
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4805 /* Find a line and append it */
38257 af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	4806 while (i < len && !BLOOM_LINEBREAK(data[i]))
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4807 i++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4808
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4809 /* Skip the line break reading CRLF as one line break */
12369 68e48bb94071 Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12329 diff changeset	4810 eol = i;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4811 if (i < len) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4812 if (data[i] == '\r' && i + 1 < len &&
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4813 data[i+1] == '\n')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4814 i += 2;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4815 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4816 i++;
12369 68e48bb94071 Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12329 diff changeset	4817 if (keepends)
68e48bb94071 Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12329 diff changeset	4818 eol = i;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4819 }
12369 68e48bb94071 Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12329 diff changeset	4820 SPLIT_APPEND(data, j, eol);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4821 j = i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4822 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4823 if (j < len) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4824 SPLIT_APPEND(data, j, len);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4825 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4826
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4827 Py_DECREF(string);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4828 return list;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4829
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4830 onError:
36877 e9a7a2784e17 SF #1444030: Fix several potential defects found by Coverity. Hye-Shik Chang <hyeshik@gmail.com> parents: 36675 diff changeset	4831 Py_XDECREF(list);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4832 Py_DECREF(string);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4833 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4834 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4835
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	4836 static
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4837 PyObject split_char(PyUnicodeObject self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4838 PyObject *list,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4839 Py_UNICODE ch,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	4840 Py_ssize_t maxcount)
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	4841 {
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	4842 register Py_ssize_t i;
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	4843 register Py_ssize_t j;
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	4844 Py_ssize_t len = self->length;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4845 PyObject *str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4846
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4847 for (i = j = 0; i < len; ) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4848 if (self->str[i] == ch) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4849 if (maxcount-- <= 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4850 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4851 SPLIT_APPEND(self->str, j, i);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4852 i = j = i + 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4853 } else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4854 i++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4855 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4856 if (j <= len) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4857 SPLIT_APPEND(self->str, j, len);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4858 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4859 return list;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4860
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4861 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4862 Py_DECREF(list);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4863 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4864 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4865
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	4866 static
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4867 PyObject split_substring(PyUnicodeObject self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4868 PyObject *list,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4869 PyUnicodeObject *substring,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	4870 Py_ssize_t maxcount)
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	4871 {
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	4872 register Py_ssize_t i;
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	4873 register Py_ssize_t j;
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	4874 Py_ssize_t len = self->length;
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	4875 Py_ssize_t sublen = substring->length;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4876 PyObject *str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4877
15609 0906c8e2c961 Fix off-by-one error in split_substring(). Fixes SF bug #122162. Guido van Rossum <guido@python.org> parents: 15579 diff changeset	4878 for (i = j = 0; i <= len - sublen; ) {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4879 if (Py_UNICODE_MATCH(self, i, substring)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4880 if (maxcount-- <= 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4881 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4882 SPLIT_APPEND(self->str, j, i);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4883 i = j = i + sublen;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4884 } else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4885 i++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4886 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4887 if (j <= len) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4888 SPLIT_APPEND(self->str, j, len);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4889 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4890 return list;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4891
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4892 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4893 Py_DECREF(list);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4894 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4895 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4896
31199 504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4897 static
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4898 PyObject rsplit_whitespace(PyUnicodeObject self,
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4899 PyObject *list,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	4900 Py_ssize_t maxcount)
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	4901 {
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	4902 register Py_ssize_t i;
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	4903 register Py_ssize_t j;
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	4904 Py_ssize_t len = self->length;
31199 504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4905 PyObject *str;
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4906
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4907 for (i = j = len - 1; i >= 0; ) {
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4908 /* find a token */
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4909 while (i >= 0 && Py_UNICODE_ISSPACE(self->str[i]))
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4910 i--;
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4911 j = i;
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4912 while (i >= 0 && !Py_UNICODE_ISSPACE(self->str[i]))
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4913 i--;
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4914 if (j > i) {
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4915 if (maxcount-- <= 0)
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4916 break;
38257 af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	4917 SPLIT_APPEND(self->str, i + 1, j + 1);
31199 504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4918 while (i >= 0 && Py_UNICODE_ISSPACE(self->str[i]))
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4919 i--;
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4920 j = i;
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4921 }
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4922 }
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4923 if (j >= 0) {
38257 af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	4924 SPLIT_APPEND(self->str, 0, j + 1);
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	4925 }
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	4926 if (PyList_Reverse(list) < 0)
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	4927 goto onError;
31199 504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4928 return list;
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4929
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4930 onError:
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4931 Py_DECREF(list);
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4932 return NULL;
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4933 }
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4934
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4935 static
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4936 PyObject rsplit_char(PyUnicodeObject self,
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4937 PyObject *list,
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4938 Py_UNICODE ch,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	4939 Py_ssize_t maxcount)
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	4940 {
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	4941 register Py_ssize_t i;
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	4942 register Py_ssize_t j;
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	4943 Py_ssize_t len = self->length;
31199 504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4944 PyObject *str;
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4945
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4946 for (i = j = len - 1; i >= 0; ) {
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4947 if (self->str[i] == ch) {
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4948 if (maxcount-- <= 0)
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4949 break;
38257 af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	4950 SPLIT_APPEND(self->str, i + 1, j + 1);
31199 504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4951 j = i = i - 1;
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4952 } else
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4953 i--;
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4954 }
31252 892116a886e0 Fix unicode.rsplit()'s bug that ignores separater on the end of string when Hye-Shik Chang <hyeshik@gmail.com> parents: 31248 diff changeset	4955 if (j >= -1) {
38257 af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	4956 SPLIT_APPEND(self->str, 0, j + 1);
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	4957 }
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	4958 if (PyList_Reverse(list) < 0)
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	4959 goto onError;
31199 504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4960 return list;
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4961
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4962 onError:
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4963 Py_DECREF(list);
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4964 return NULL;
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4965 }
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4966
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4967 static
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4968 PyObject rsplit_substring(PyUnicodeObject self,
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4969 PyObject *list,
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4970 PyUnicodeObject *substring,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	4971 Py_ssize_t maxcount)
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	4972 {
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	4973 register Py_ssize_t i;
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	4974 register Py_ssize_t j;
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	4975 Py_ssize_t len = self->length;
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	4976 Py_ssize_t sublen = substring->length;
31199 504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4977 PyObject *str;
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4978
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4979 for (i = len - sublen, j = len; i >= 0; ) {
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4980 if (Py_UNICODE_MATCH(self, i, substring)) {
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4981 if (maxcount-- <= 0)
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4982 break;
38257 af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	4983 SPLIT_APPEND(self->str, i + sublen, j);
31199 504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4984 j = i;
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4985 i -= sublen;
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4986 } else
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4987 i--;
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4988 }
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4989 if (j >= 0) {
38257 af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	4990 SPLIT_APPEND(self->str, 0, j);
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	4991 }
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	4992 if (PyList_Reverse(list) < 0)
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	4993 goto onError;
31199 504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4994 return list;
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4995
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4996 onError:
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4997 Py_DECREF(list);
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4998 return NULL;
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	4999 }
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	5000
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5001 #undef SPLIT_APPEND
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5002
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5003 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5004 PyObject split(PyUnicodeObject self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5005 PyUnicodeObject *substring,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	5006 Py_ssize_t maxcount)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5007 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5008 PyObject *list;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5009
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5010 if (maxcount < 0)
37662 0678ce9fb6b2 Change more ints to Py_ssize_t. Martin v. Löwis <martin@v.loewis.de> parents: 37661 diff changeset	5011 maxcount = PY_SSIZE_T_MAX;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5012
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5013 list = PyList_New(0);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5014 if (!list)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5015 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5016
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5017 if (substring == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5018 return split_whitespace(self,list,maxcount);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5019
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5020 else if (substring->length == 1)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5021 return split_char(self,list,substring->str[0],maxcount);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5022
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5023 else if (substring->length == 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5024 Py_DECREF(list);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5025 PyErr_SetString(PyExc_ValueError, "empty separator");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5026 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5027 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5028 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5029 return split_substring(self,list,substring,maxcount);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5030 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5031
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	5032 static
31199 504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	5033 PyObject rsplit(PyUnicodeObject self,
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	5034 PyUnicodeObject *substring,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	5035 Py_ssize_t maxcount)
31199 504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	5036 {
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	5037 PyObject *list;
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	5038
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	5039 if (maxcount < 0)
37662 0678ce9fb6b2 Change more ints to Py_ssize_t. Martin v. Löwis <martin@v.loewis.de> parents: 37661 diff changeset	5040 maxcount = PY_SSIZE_T_MAX;
31199 504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	5041
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	5042 list = PyList_New(0);
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	5043 if (!list)
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	5044 return NULL;
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	5045
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	5046 if (substring == NULL)
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	5047 return rsplit_whitespace(self,list,maxcount);
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	5048
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	5049 else if (substring->length == 1)
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	5050 return rsplit_char(self,list,substring->str[0],maxcount);
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	5051
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	5052 else if (substring->length == 0) {
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	5053 Py_DECREF(list);
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	5054 PyErr_SetString(PyExc_ValueError, "empty separator");
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	5055 return NULL;
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	5056 }
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	5057 else
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	5058 return rsplit_substring(self,list,substring,maxcount);
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	5059 }
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	5060
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	5061 static
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5062 PyObject replace(PyUnicodeObject self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5063 PyUnicodeObject *str1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5064 PyUnicodeObject *str2,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	5065 Py_ssize_t maxcount)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5066 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5067 PyUnicodeObject *u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5068
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5069 if (maxcount < 0)
37662 0678ce9fb6b2 Change more ints to Py_ssize_t. Martin v. Löwis <martin@v.loewis.de> parents: 37661 diff changeset	5070 maxcount = PY_SSIZE_T_MAX;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5071
38278 260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5072 if (str1->length == str2->length) {
260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5073 /* same length */
37662 0678ce9fb6b2 Change more ints to Py_ssize_t. Martin v. Löwis <martin@v.loewis.de> parents: 37661 diff changeset	5074 Py_ssize_t i;
38278 260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5075 if (str1->length == 1) {
260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5076 /* replace characters */
260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5077 Py_UNICODE u1, u2;
260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5078 if (!findchar(self->str, self->length, str1->str[0]))
260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5079 goto nothing;
260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5080 u = (PyUnicodeObject*) PyUnicode_FromUnicode(NULL, self->length);
260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5081 if (!u)
260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5082 return NULL;
260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5083 Py_UNICODE_COPY(u->str, self->str, self->length);
260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5084 u1 = str1->str[0];
260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5085 u2 = str2->str[0];
260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5086 for (i = 0; i < u->length; i++)
260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5087 if (u->str[i] == u1) {
260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5088 if (--maxcount < 0)
260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5089 break;
260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5090 u->str[i] = u2;
260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5091 }
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5092 } else {
38278 260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5093 i = fastsearch(
260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5094 self->str, self->length, str1->str, str1->length, FAST_SEARCH
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5095 );
38278 260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5096 if (i < 0)
260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5097 goto nothing;
260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5098 u = (PyUnicodeObject*) PyUnicode_FromUnicode(NULL, self->length);
260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5099 if (!u)
260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5100 return NULL;
260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5101 Py_UNICODE_COPY(u->str, self->str, self->length);
260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5102 while (i <= self->length - str1->length)
260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5103 if (Py_UNICODE_MATCH(self, i, str1)) {
260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5104 if (--maxcount < 0)
260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5105 break;
260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5106 Py_UNICODE_COPY(u->str+i, str2->str, str2->length);
260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5107 i += str1->length;
260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5108 } else
260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5109 i++;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5110 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5111 } else {
38278 260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5112
38440 5b0753b9d16a needforspeed: replace improvements, changed to Py_LOCAL_INLINE Fredrik Lundh <fredrik@pythonware.com> parents: 38427 diff changeset	5113 Py_ssize_t n, i, j, e;
38299 ae6bbf2f2a41 needforspeed: check for overflow in replace (from Andrew Dalke) Fredrik Lundh <fredrik@pythonware.com> parents: 38278 diff changeset	5114 Py_ssize_t product, new_size, delta;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5115 Py_UNICODE *p;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5116
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5117 /* replace strings */
38400 33c328896f52 needforspeed: stringlib refactoring, continued. added count and Fredrik Lundh <fredrik@pythonware.com> parents: 38391 diff changeset	5118 n = stringlib_count(self->str, self->length, str1->str, str1->length);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5119 if (n > maxcount)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5120 n = maxcount;
38278 260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5121 if (n == 0)
260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5122 goto nothing;
38299 ae6bbf2f2a41 needforspeed: check for overflow in replace (from Andrew Dalke) Fredrik Lundh <fredrik@pythonware.com> parents: 38278 diff changeset	5123 /* new_size = self->length + n * (str2->length - str1->length)); */
ae6bbf2f2a41 needforspeed: check for overflow in replace (from Andrew Dalke) Fredrik Lundh <fredrik@pythonware.com> parents: 38278 diff changeset	5124 delta = (str2->length - str1->length);
ae6bbf2f2a41 needforspeed: check for overflow in replace (from Andrew Dalke) Fredrik Lundh <fredrik@pythonware.com> parents: 38278 diff changeset	5125 if (delta == 0) {
ae6bbf2f2a41 needforspeed: check for overflow in replace (from Andrew Dalke) Fredrik Lundh <fredrik@pythonware.com> parents: 38278 diff changeset	5126 new_size = self->length;
ae6bbf2f2a41 needforspeed: check for overflow in replace (from Andrew Dalke) Fredrik Lundh <fredrik@pythonware.com> parents: 38278 diff changeset	5127 } else {
ae6bbf2f2a41 needforspeed: check for overflow in replace (from Andrew Dalke) Fredrik Lundh <fredrik@pythonware.com> parents: 38278 diff changeset	5128 product = n * (str2->length - str1->length);
ae6bbf2f2a41 needforspeed: check for overflow in replace (from Andrew Dalke) Fredrik Lundh <fredrik@pythonware.com> parents: 38278 diff changeset	5129 if ((product / (str2->length - str1->length)) != n) {
ae6bbf2f2a41 needforspeed: check for overflow in replace (from Andrew Dalke) Fredrik Lundh <fredrik@pythonware.com> parents: 38278 diff changeset	5130 PyErr_SetString(PyExc_OverflowError,
ae6bbf2f2a41 needforspeed: check for overflow in replace (from Andrew Dalke) Fredrik Lundh <fredrik@pythonware.com> parents: 38278 diff changeset	5131 "replace string is too long");
ae6bbf2f2a41 needforspeed: check for overflow in replace (from Andrew Dalke) Fredrik Lundh <fredrik@pythonware.com> parents: 38278 diff changeset	5132 return NULL;
ae6bbf2f2a41 needforspeed: check for overflow in replace (from Andrew Dalke) Fredrik Lundh <fredrik@pythonware.com> parents: 38278 diff changeset	5133 }
ae6bbf2f2a41 needforspeed: check for overflow in replace (from Andrew Dalke) Fredrik Lundh <fredrik@pythonware.com> parents: 38278 diff changeset	5134 new_size = self->length + product;
ae6bbf2f2a41 needforspeed: check for overflow in replace (from Andrew Dalke) Fredrik Lundh <fredrik@pythonware.com> parents: 38278 diff changeset	5135 if (new_size < 0) {
ae6bbf2f2a41 needforspeed: check for overflow in replace (from Andrew Dalke) Fredrik Lundh <fredrik@pythonware.com> parents: 38278 diff changeset	5136 PyErr_SetString(PyExc_OverflowError,
ae6bbf2f2a41 needforspeed: check for overflow in replace (from Andrew Dalke) Fredrik Lundh <fredrik@pythonware.com> parents: 38278 diff changeset	5137 "replace string is too long");
ae6bbf2f2a41 needforspeed: check for overflow in replace (from Andrew Dalke) Fredrik Lundh <fredrik@pythonware.com> parents: 38278 diff changeset	5138 return NULL;
ae6bbf2f2a41 needforspeed: check for overflow in replace (from Andrew Dalke) Fredrik Lundh <fredrik@pythonware.com> parents: 38278 diff changeset	5139 }
ae6bbf2f2a41 needforspeed: check for overflow in replace (from Andrew Dalke) Fredrik Lundh <fredrik@pythonware.com> parents: 38278 diff changeset	5140 }
ae6bbf2f2a41 needforspeed: check for overflow in replace (from Andrew Dalke) Fredrik Lundh <fredrik@pythonware.com> parents: 38278 diff changeset	5141 u = _PyUnicode_New(new_size);
38278 260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5142 if (!u)
260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5143 return NULL;
260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5144 i = 0;
260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5145 p = u->str;
38440 5b0753b9d16a needforspeed: replace improvements, changed to Py_LOCAL_INLINE Fredrik Lundh <fredrik@pythonware.com> parents: 38427 diff changeset	5146 e = self->length - str1->length;
38278 260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5147 if (str1->length > 0) {
38440 5b0753b9d16a needforspeed: replace improvements, changed to Py_LOCAL_INLINE Fredrik Lundh <fredrik@pythonware.com> parents: 38427 diff changeset	5148 while (n-- > 0) {
5b0753b9d16a needforspeed: replace improvements, changed to Py_LOCAL_INLINE Fredrik Lundh <fredrik@pythonware.com> parents: 38427 diff changeset	5149 /* look for next match */
5b0753b9d16a needforspeed: replace improvements, changed to Py_LOCAL_INLINE Fredrik Lundh <fredrik@pythonware.com> parents: 38427 diff changeset	5150 j = i;
5b0753b9d16a needforspeed: replace improvements, changed to Py_LOCAL_INLINE Fredrik Lundh <fredrik@pythonware.com> parents: 38427 diff changeset	5151 while (j <= e) {
5b0753b9d16a needforspeed: replace improvements, changed to Py_LOCAL_INLINE Fredrik Lundh <fredrik@pythonware.com> parents: 38427 diff changeset	5152 if (Py_UNICODE_MATCH(self, j, str1))
5b0753b9d16a needforspeed: replace improvements, changed to Py_LOCAL_INLINE Fredrik Lundh <fredrik@pythonware.com> parents: 38427 diff changeset	5153 break;
5b0753b9d16a needforspeed: replace improvements, changed to Py_LOCAL_INLINE Fredrik Lundh <fredrik@pythonware.com> parents: 38427 diff changeset	5154 j++;
5b0753b9d16a needforspeed: replace improvements, changed to Py_LOCAL_INLINE Fredrik Lundh <fredrik@pythonware.com> parents: 38427 diff changeset	5155 }
5b0753b9d16a needforspeed: replace improvements, changed to Py_LOCAL_INLINE Fredrik Lundh <fredrik@pythonware.com> parents: 38427 diff changeset	5156 if (j > i) {
5b0753b9d16a needforspeed: replace improvements, changed to Py_LOCAL_INLINE Fredrik Lundh <fredrik@pythonware.com> parents: 38427 diff changeset	5157 if (j > e)
5b0753b9d16a needforspeed: replace improvements, changed to Py_LOCAL_INLINE Fredrik Lundh <fredrik@pythonware.com> parents: 38427 diff changeset	5158 break;
5b0753b9d16a needforspeed: replace improvements, changed to Py_LOCAL_INLINE Fredrik Lundh <fredrik@pythonware.com> parents: 38427 diff changeset	5159 /* copy unchanged part [i:j] */
5b0753b9d16a needforspeed: replace improvements, changed to Py_LOCAL_INLINE Fredrik Lundh <fredrik@pythonware.com> parents: 38427 diff changeset	5160 Py_UNICODE_COPY(p, self->str+i, j-i);
5b0753b9d16a needforspeed: replace improvements, changed to Py_LOCAL_INLINE Fredrik Lundh <fredrik@pythonware.com> parents: 38427 diff changeset	5161 p += j - i;
5b0753b9d16a needforspeed: replace improvements, changed to Py_LOCAL_INLINE Fredrik Lundh <fredrik@pythonware.com> parents: 38427 diff changeset	5162 }
5b0753b9d16a needforspeed: replace improvements, changed to Py_LOCAL_INLINE Fredrik Lundh <fredrik@pythonware.com> parents: 38427 diff changeset	5163 /* copy substitution string */
5b0753b9d16a needforspeed: replace improvements, changed to Py_LOCAL_INLINE Fredrik Lundh <fredrik@pythonware.com> parents: 38427 diff changeset	5164 if (str2->length > 0) {
38278 260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5165 Py_UNICODE_COPY(p, str2->str, str2->length);
260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5166 p += str2->length;
38440 5b0753b9d16a needforspeed: replace improvements, changed to Py_LOCAL_INLINE Fredrik Lundh <fredrik@pythonware.com> parents: 38427 diff changeset	5167 }
5b0753b9d16a needforspeed: replace improvements, changed to Py_LOCAL_INLINE Fredrik Lundh <fredrik@pythonware.com> parents: 38427 diff changeset	5168 i = j + str1->length;
5b0753b9d16a needforspeed: replace improvements, changed to Py_LOCAL_INLINE Fredrik Lundh <fredrik@pythonware.com> parents: 38427 diff changeset	5169 }
5b0753b9d16a needforspeed: replace improvements, changed to Py_LOCAL_INLINE Fredrik Lundh <fredrik@pythonware.com> parents: 38427 diff changeset	5170 if (i < self->length)
5b0753b9d16a needforspeed: replace improvements, changed to Py_LOCAL_INLINE Fredrik Lundh <fredrik@pythonware.com> parents: 38427 diff changeset	5171 /* copy tail [i:] */
5b0753b9d16a needforspeed: replace improvements, changed to Py_LOCAL_INLINE Fredrik Lundh <fredrik@pythonware.com> parents: 38427 diff changeset	5172 Py_UNICODE_COPY(p, self->str+i, self->length-i);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5173 } else {
38440 5b0753b9d16a needforspeed: replace improvements, changed to Py_LOCAL_INLINE Fredrik Lundh <fredrik@pythonware.com> parents: 38427 diff changeset	5174 /* interleave */
38278 260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5175 while (n > 0) {
260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5176 Py_UNICODE_COPY(p, str2->str, str2->length);
260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5177 p += str2->length;
260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5178 if (--n <= 0)
260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5179 break;
260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5180 *p++ = self->str[i++];
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5181 }
38278 260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5182 Py_UNICODE_COPY(p, self->str+i, self->length-i);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5183 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5184 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5185 return (PyObject *) u;
38278 260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5186
260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5187 nothing:
260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5188 /* nothing to replace; return original string (when possible) */
260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5189 if (PyUnicode_CheckExact(self)) {
260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5190 Py_INCREF(self);
260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5191 return (PyObject *) self;
260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5192 }
260d5a6ce30f needforspeed: refactored the replace code slightly; special-case Fredrik Lundh <fredrik@pythonware.com> parents: 38276 diff changeset	5193 return PyUnicode_FromUnicode(self->str, self->length);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5194 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5195
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5196 /* --- Unicode Object Methods --------------------------------------------- */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5197
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5198 PyDoc_STRVAR(title__doc__,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5199 "S.title() -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5200 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5201 Return a titlecased version of S, i.e. words start with title case\n\
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5202 characters, all remaining cased characters have lower case.");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5203
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5204 static PyObject*
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	5205 unicode_title(PyUnicodeObject *self)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5206 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5207 return fixup(self, fixtitle);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5208 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5209
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5210 PyDoc_STRVAR(capitalize__doc__,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5211 "S.capitalize() -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5212 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5213 Return a capitalized version of S, i.e. make the first character\n\
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5214 have upper case.");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5215
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5216 static PyObject*
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	5217 unicode_capitalize(PyUnicodeObject *self)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5218 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5219 return fixup(self, fixcapitalize);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5220 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5221
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5222 #if 0
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5223 PyDoc_STRVAR(capwords__doc__,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5224 "S.capwords() -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5225 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5226 Apply .capitalize() to all words in S and return the result with\n\
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5227 normalized whitespace (all whitespace strings are replaced by ' ').");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5228
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5229 static PyObject*
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	5230 unicode_capwords(PyUnicodeObject *self)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5231 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5232 PyObject *list;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5233 PyObject *item;
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	5234 Py_ssize_t i;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5235
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5236 /* Split into words */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5237 list = split(self, NULL, -1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5238 if (!list)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5239 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5240
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5241 /* Capitalize each word */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5242 for (i = 0; i < PyList_GET_SIZE(list); i++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5243 item = fixup((PyUnicodeObject *)PyList_GET_ITEM(list, i),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5244 fixcapitalize);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5245 if (item == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5246 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5247 Py_DECREF(PyList_GET_ITEM(list, i));
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5248 PyList_SET_ITEM(list, i, item);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5249 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5250
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5251 /* Join the words to form a new string */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5252 item = PyUnicode_Join(NULL, list);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5253
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5254 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5255 Py_DECREF(list);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5256 return (PyObject *)item;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5257 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5258 #endif
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5259
31021 21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods. Raymond Hettinger <python@rcn.com> parents: 30765 diff changeset	5260 /* Argument converter. Coerces to a single unicode character */
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods. Raymond Hettinger <python@rcn.com> parents: 30765 diff changeset	5261
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods. Raymond Hettinger <python@rcn.com> parents: 30765 diff changeset	5262 static int
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods. Raymond Hettinger <python@rcn.com> parents: 30765 diff changeset	5263 convert_uc(PyObject obj, void addr)
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods. Raymond Hettinger <python@rcn.com> parents: 30765 diff changeset	5264 {
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods. Raymond Hettinger <python@rcn.com> parents: 30765 diff changeset	5265 Py_UNICODE fillcharloc = (Py_UNICODE )addr;
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods. Raymond Hettinger <python@rcn.com> parents: 30765 diff changeset	5266 PyObject *uniobj;
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods. Raymond Hettinger <python@rcn.com> parents: 30765 diff changeset	5267 Py_UNICODE *unistr;
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods. Raymond Hettinger <python@rcn.com> parents: 30765 diff changeset	5268
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods. Raymond Hettinger <python@rcn.com> parents: 30765 diff changeset	5269 uniobj = PyUnicode_FromObject(obj);
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods. Raymond Hettinger <python@rcn.com> parents: 30765 diff changeset	5270 if (uniobj == NULL) {
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods. Raymond Hettinger <python@rcn.com> parents: 30765 diff changeset	5271 PyErr_SetString(PyExc_TypeError,
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods. Raymond Hettinger <python@rcn.com> parents: 30765 diff changeset	5272 "The fill character cannot be converted to Unicode");
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods. Raymond Hettinger <python@rcn.com> parents: 30765 diff changeset	5273 return 0;
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods. Raymond Hettinger <python@rcn.com> parents: 30765 diff changeset	5274 }
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods. Raymond Hettinger <python@rcn.com> parents: 30765 diff changeset	5275 if (PyUnicode_GET_SIZE(uniobj) != 1) {
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods. Raymond Hettinger <python@rcn.com> parents: 30765 diff changeset	5276 PyErr_SetString(PyExc_TypeError,
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods. Raymond Hettinger <python@rcn.com> parents: 30765 diff changeset	5277 "The fill character must be exactly one character long");
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods. Raymond Hettinger <python@rcn.com> parents: 30765 diff changeset	5278 Py_DECREF(uniobj);
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods. Raymond Hettinger <python@rcn.com> parents: 30765 diff changeset	5279 return 0;
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods. Raymond Hettinger <python@rcn.com> parents: 30765 diff changeset	5280 }
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods. Raymond Hettinger <python@rcn.com> parents: 30765 diff changeset	5281 unistr = PyUnicode_AS_UNICODE(uniobj);
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods. Raymond Hettinger <python@rcn.com> parents: 30765 diff changeset	5282 *fillcharloc = unistr[0];
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods. Raymond Hettinger <python@rcn.com> parents: 30765 diff changeset	5283 Py_DECREF(uniobj);
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods. Raymond Hettinger <python@rcn.com> parents: 30765 diff changeset	5284 return 1;
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods. Raymond Hettinger <python@rcn.com> parents: 30765 diff changeset	5285 }
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods. Raymond Hettinger <python@rcn.com> parents: 30765 diff changeset	5286
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5287 PyDoc_STRVAR(center__doc__,
31021 21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods. Raymond Hettinger <python@rcn.com> parents: 30765 diff changeset	5288 "S.center(width[, fillchar]) -> unicode\n\
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5289 \n\
31021 21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods. Raymond Hettinger <python@rcn.com> parents: 30765 diff changeset	5290 Return S centered in a Unicode string of length width. Padding is\n\
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods. Raymond Hettinger <python@rcn.com> parents: 30765 diff changeset	5291 done using the specified fill character (default is a space)");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5292
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5293 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5294 unicode_center(PyUnicodeObject self, PyObject args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5295 {
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	5296 Py_ssize_t marg, left;
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	5297 Py_ssize_t width;
31021 21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods. Raymond Hettinger <python@rcn.com> parents: 30765 diff changeset	5298 Py_UNICODE fillchar = ' ';
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods. Raymond Hettinger <python@rcn.com> parents: 30765 diff changeset	5299
36532 0c096be27000 Use correct PyArg_Parse format char for Py_ssize_t in unicode.center(). Thomas Wouters <thomas@python.org> parents: 36519 diff changeset	5300 if (!PyArg_ParseTuple(args, "n\|O&:center", &width, convert_uc, &fillchar))
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5301 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5302
19885 72e6eb653312 More on bug 460020: disable many optimizations of unicode subclasses. Tim Peters <tim.peters@gmail.com> parents: 19849 diff changeset	5303 if (self->length >= width && PyUnicode_CheckExact(self)) {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5304 Py_INCREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5305 return (PyObject*) self;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5306 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5307
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5308 marg = width - self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5309 left = marg / 2 + (marg & width & 1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5310
31021 21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods. Raymond Hettinger <python@rcn.com> parents: 30765 diff changeset	5311 return (PyObject*) pad(self, left, marg - left, fillchar);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5312 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5313
13890 8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	5314 #if 0
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	5315
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	5316 /* This code should go into some future Unicode collation support
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	5317 module. The basic comparison should compare ordinals on a naive
13946 7a99450da2d6 Clean up warning from Monterey compiler. Trent Mick <trentm@activestate.com> parents: 13890 diff changeset	5318 basis (this is what Java does and thus JPython too). */
13890 8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	5319
13337 e101600adee4 Bill Tutt: Marc-André Lemburg <mal@egenix.com> parents: 13209 diff changeset	5320 /* speedy UTF-16 code point order comparison */
e101600adee4 Bill Tutt: Marc-André Lemburg <mal@egenix.com> parents: 13209 diff changeset	5321 /* gleaned from: */
e101600adee4 Bill Tutt: Marc-André Lemburg <mal@egenix.com> parents: 13209 diff changeset	5322 /* http://www-4.ibm.com/software/developer/library/utf16.html?dwzone=unicode */
e101600adee4 Bill Tutt: Marc-André Lemburg <mal@egenix.com> parents: 13209 diff changeset	5323
13411 9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	5324 static short utf16Fixup[32] =
13337 e101600adee4 Bill Tutt: Marc-André Lemburg <mal@egenix.com> parents: 13209 diff changeset	5325 {
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	5326 0, 0, 0, 0, 0, 0, 0, 0,
13337 e101600adee4 Bill Tutt: Marc-André Lemburg <mal@egenix.com> parents: 13209 diff changeset	5327 0, 0, 0, 0, 0, 0, 0, 0,
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	5328 0, 0, 0, 0, 0, 0, 0, 0,
13411 9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	5329 0, 0, 0, 0x2000, -0x800, -0x800, -0x800, -0x800
13337 e101600adee4 Bill Tutt: Marc-André Lemburg <mal@egenix.com> parents: 13209 diff changeset	5330 };
e101600adee4 Bill Tutt: Marc-André Lemburg <mal@egenix.com> parents: 13209 diff changeset	5331
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5332 static int
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5333 unicode_compare(PyUnicodeObject str1, PyUnicodeObject str2)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5334 {
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	5335 Py_ssize_t len1, len2;
13337 e101600adee4 Bill Tutt: Marc-André Lemburg <mal@egenix.com> parents: 13209 diff changeset	5336
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5337 Py_UNICODE *s1 = str1->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5338 Py_UNICODE *s2 = str2->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5339
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5340 len1 = str1->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5341 len2 = str2->length;
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	5342
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5343 while (len1 > 0 && len2 > 0) {
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	5344 Py_UNICODE c1, c2;
13337 e101600adee4 Bill Tutt: Marc-André Lemburg <mal@egenix.com> parents: 13209 diff changeset	5345
e101600adee4 Bill Tutt: Marc-André Lemburg <mal@egenix.com> parents: 13209 diff changeset	5346 c1 = *s1++;
e101600adee4 Bill Tutt: Marc-André Lemburg <mal@egenix.com> parents: 13209 diff changeset	5347 c2 = *s2++;
18279 8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case Fredrik Lundh <fredrik@pythonware.com> parents: 18278 diff changeset	5348
13337 e101600adee4 Bill Tutt: Marc-André Lemburg <mal@egenix.com> parents: 13209 diff changeset	5349 if (c1 > (1<<11) * 26)
e101600adee4 Bill Tutt: Marc-André Lemburg <mal@egenix.com> parents: 13209 diff changeset	5350 c1 += utf16Fixup[c1>>11];
e101600adee4 Bill Tutt: Marc-André Lemburg <mal@egenix.com> parents: 13209 diff changeset	5351 if (c2 > (1<<11) * 26)
e101600adee4 Bill Tutt: Marc-André Lemburg <mal@egenix.com> parents: 13209 diff changeset	5352 c2 += utf16Fixup[c2>>11];
18279 8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case Fredrik Lundh <fredrik@pythonware.com> parents: 18278 diff changeset	5353 /* now c1 and c2 are in UTF-32-compatible order */
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case Fredrik Lundh <fredrik@pythonware.com> parents: 18278 diff changeset	5354
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case Fredrik Lundh <fredrik@pythonware.com> parents: 18278 diff changeset	5355 if (c1 != c2)
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case Fredrik Lundh <fredrik@pythonware.com> parents: 18278 diff changeset	5356 return (c1 < c2) ? -1 : 1;
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	5357
13337 e101600adee4 Bill Tutt: Marc-André Lemburg <mal@egenix.com> parents: 13209 diff changeset	5358 len1--; len2--;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5359 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5360
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5361 return (len1 < len2) ? -1 : (len1 != len2);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5362 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5363
13890 8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	5364 #else
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	5365
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	5366 static int
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	5367 unicode_compare(PyUnicodeObject str1, PyUnicodeObject str2)
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	5368 {
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	5369 register Py_ssize_t len1, len2;
13890 8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	5370
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	5371 Py_UNICODE *s1 = str1->str;
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	5372 Py_UNICODE *s2 = str2->str;
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	5373
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	5374 len1 = str1->length;
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	5375 len2 = str2->length;
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	5376
13890 8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	5377 while (len1 > 0 && len2 > 0) {
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	5378 Py_UNICODE c1, c2;
18279 8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case Fredrik Lundh <fredrik@pythonware.com> parents: 18278 diff changeset	5379
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case Fredrik Lundh <fredrik@pythonware.com> parents: 18278 diff changeset	5380 c1 = *s1++;
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case Fredrik Lundh <fredrik@pythonware.com> parents: 18278 diff changeset	5381 c2 = *s2++;
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case Fredrik Lundh <fredrik@pythonware.com> parents: 18278 diff changeset	5382
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case Fredrik Lundh <fredrik@pythonware.com> parents: 18278 diff changeset	5383 if (c1 != c2)
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case Fredrik Lundh <fredrik@pythonware.com> parents: 18278 diff changeset	5384 return (c1 < c2) ? -1 : 1;
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case Fredrik Lundh <fredrik@pythonware.com> parents: 18278 diff changeset	5385
13890 8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	5386 len1--; len2--;
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	5387 }
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	5388
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	5389 return (len1 < len2) ? -1 : (len1 != len2);
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	5390 }
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	5391
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	5392 #endif
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	5393
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5394 int PyUnicode_Compare(PyObject *left,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5395 PyObject *right)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5396 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5397 PyUnicodeObject u = NULL, v = NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5398 int result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5399
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5400 /* Coerce the two arguments */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5401 u = (PyUnicodeObject *)PyUnicode_FromObject(left);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5402 if (u == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5403 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5404 v = (PyUnicodeObject *)PyUnicode_FromObject(right);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5405 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5406 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5407
13620 5b727ab37092 Spelling fixes supplied by Rob W. W. Hooft. All these are fixes in either Thomas Wouters <thomas@python.org> parents: 13576 diff changeset	5408 /* Shortcut for empty or interned objects */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5409 if (v == u) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5410 Py_DECREF(u);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5411 Py_DECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5412 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5413 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5414
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5415 result = unicode_compare(u, v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5416
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5417 Py_DECREF(u);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5418 Py_DECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5419 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5420
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5421 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5422 Py_XDECREF(u);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5423 Py_XDECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5424 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5425 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5426
39487 635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5427 PyObject PyUnicode_RichCompare(PyObject left,
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5428 PyObject *right,
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5429 int op)
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5430 {
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5431 int result;
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5432
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5433 result = PyUnicode_Compare(left, right);
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5434 if (result == -1 && PyErr_Occurred())
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5435 goto onError;
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5436
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5437 /* Convert the return value to a Boolean */
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5438 switch (op) {
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5439 case Py_EQ:
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5440 result = (result == 0);
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5441 break;
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5442 case Py_NE:
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5443 result = (result != 0);
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5444 break;
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5445 case Py_LE:
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5446 result = (result <= 0);
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5447 break;
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5448 case Py_GE:
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5449 result = (result >= 0);
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5450 break;
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5451 case Py_LT:
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5452 result = (result == -1);
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5453 break;
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5454 case Py_GT:
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5455 result = (result == 1);
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5456 break;
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5457 }
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5458 return PyBool_FromLong(result);
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5459
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5460 onError:
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5461
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5462 /* Standard case
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5463
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5464 Type errors mean that PyUnicode_FromObject() could not convert
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5465 one of the arguments (usually the right hand side) to Unicode,
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5466 ie. we can't handle the comparison request. However, it is
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5467 possible that the other object knows a comparison method, which
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5468 is why we return Py_NotImplemented to give the other object a
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5469 chance.
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5470
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5471 */
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5472 if (PyErr_ExceptionMatches(PyExc_TypeError)) {
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5473 PyErr_Clear();
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5474 Py_INCREF(Py_NotImplemented);
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5475 return Py_NotImplemented;
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5476 }
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5477 if (op != Py_EQ && op != Py_NE)
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5478 return NULL;
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5479
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5480 /* Equality comparison.
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5481
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5482 This is a special case: we silence any PyExc_UnicodeDecodeError
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5483 and instead turn it into a PyErr_UnicodeWarning.
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5484
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5485 */
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5486 if (!PyErr_ExceptionMatches(PyExc_UnicodeDecodeError))
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5487 return NULL;
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5488 PyErr_Clear();
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5489 if (PyErr_Warn(PyExc_UnicodeWarning,
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5490 (op == Py_EQ) ?
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5491 "Unicode equal comparison "
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5492 "failed to convert both arguments to Unicode - "
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5493 "interpreting them as being unequal" :
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5494 "Unicode unequal comparison "
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5495 "failed to convert both arguments to Unicode - "
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5496 "interpreting them as being unequal"
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5497 ) < 0)
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5498 return NULL;
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5499 result = (op == Py_NE);
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5500 return PyBool_FromLong(result);
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5501 }
635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	5502
11997 38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	5503 int PyUnicode_Contains(PyObject *container,
38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	5504 PyObject *element)
38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	5505 {
38424 3d81faf47af7 needforspeed: more stringlib refactoring Fredrik Lundh <fredrik@pythonware.com> parents: 38422 diff changeset	5506 PyObject str, sub;
3d81faf47af7 needforspeed: more stringlib refactoring Fredrik Lundh <fredrik@pythonware.com> parents: 38422 diff changeset	5507 int result;
11997 38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	5508
38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	5509 /* Coerce the two arguments */
38424 3d81faf47af7 needforspeed: more stringlib refactoring Fredrik Lundh <fredrik@pythonware.com> parents: 38422 diff changeset	5510 sub = PyUnicode_FromObject(element);
3d81faf47af7 needforspeed: more stringlib refactoring Fredrik Lundh <fredrik@pythonware.com> parents: 38422 diff changeset	5511 if (!sub) {
13049 0be4b7b650e3 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12957 diff changeset	5512 PyErr_SetString(PyExc_TypeError,
24598 12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a Barry Warsaw <barry@python.org> parents: 24392 diff changeset	5513 "'in <string>' requires string as left operand");
38243 0ba711a496e6 needforspeed: fixed unicode "in" operator to use same implementation Fredrik Lundh <fredrik@pythonware.com> parents: 38241 diff changeset	5514 return -1;
0ba711a496e6 needforspeed: fixed unicode "in" operator to use same implementation Fredrik Lundh <fredrik@pythonware.com> parents: 38241 diff changeset	5515 }
0ba711a496e6 needforspeed: fixed unicode "in" operator to use same implementation Fredrik Lundh <fredrik@pythonware.com> parents: 38241 diff changeset	5516
38424 3d81faf47af7 needforspeed: more stringlib refactoring Fredrik Lundh <fredrik@pythonware.com> parents: 38422 diff changeset	5517 str = PyUnicode_FromObject(container);
3d81faf47af7 needforspeed: more stringlib refactoring Fredrik Lundh <fredrik@pythonware.com> parents: 38422 diff changeset	5518 if (!str) {
3d81faf47af7 needforspeed: more stringlib refactoring Fredrik Lundh <fredrik@pythonware.com> parents: 38422 diff changeset	5519 Py_DECREF(sub);
38243 0ba711a496e6 needforspeed: fixed unicode "in" operator to use same implementation Fredrik Lundh <fredrik@pythonware.com> parents: 38241 diff changeset	5520 return -1;
0ba711a496e6 needforspeed: fixed unicode "in" operator to use same implementation Fredrik Lundh <fredrik@pythonware.com> parents: 38241 diff changeset	5521 }
11997 38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	5522
38424 3d81faf47af7 needforspeed: more stringlib refactoring Fredrik Lundh <fredrik@pythonware.com> parents: 38422 diff changeset	5523 result = stringlib_contains_obj(str, sub);
3d81faf47af7 needforspeed: more stringlib refactoring Fredrik Lundh <fredrik@pythonware.com> parents: 38422 diff changeset	5524
3d81faf47af7 needforspeed: more stringlib refactoring Fredrik Lundh <fredrik@pythonware.com> parents: 38422 diff changeset	5525 Py_DECREF(str);
3d81faf47af7 needforspeed: more stringlib refactoring Fredrik Lundh <fredrik@pythonware.com> parents: 38422 diff changeset	5526 Py_DECREF(sub);
3d81faf47af7 needforspeed: more stringlib refactoring Fredrik Lundh <fredrik@pythonware.com> parents: 38422 diff changeset	5527
3d81faf47af7 needforspeed: more stringlib refactoring Fredrik Lundh <fredrik@pythonware.com> parents: 38422 diff changeset	5528 return result;
11997 38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	5529 }
38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	5530
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5531 /* Concat to string or Unicode object giving a new Unicode object. */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5532
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5533 PyObject PyUnicode_Concat(PyObject left,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5534 PyObject *right)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5535 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5536 PyUnicodeObject u = NULL, v = NULL, *w;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5537
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5538 /* Coerce the two arguments */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5539 u = (PyUnicodeObject *)PyUnicode_FromObject(left);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5540 if (u == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5541 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5542 v = (PyUnicodeObject *)PyUnicode_FromObject(right);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5543 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5544 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5545
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5546 /* Shortcuts */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5547 if (v == unicode_empty) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5548 Py_DECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5549 return (PyObject *)u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5550 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5551 if (u == unicode_empty) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5552 Py_DECREF(u);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5553 return (PyObject *)v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5554 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5555
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5556 /* Concat the two Unicode strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5557 w = _PyUnicode_New(u->length + v->length);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5558 if (w == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5559 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5560 Py_UNICODE_COPY(w->str, u->str, u->length);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5561 Py_UNICODE_COPY(w->str + u->length, v->str, v->length);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5562
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5563 Py_DECREF(u);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5564 Py_DECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5565 return (PyObject *)w;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5566
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5567 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5568 Py_XDECREF(u);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5569 Py_XDECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5570 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5571 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5572
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5573 PyDoc_STRVAR(count__doc__,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5574 "S.count(sub[, start[, end]]) -> int\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5575 \n\
38236 7e16fcce34a1 docstring tweaks: count counts non-overlapping substrings, not Fredrik Lundh <fredrik@pythonware.com> parents: 38182 diff changeset	5576 Return the number of non-overlapping occurrences of substring sub in\n\
7e16fcce34a1 docstring tweaks: count counts non-overlapping substrings, not Fredrik Lundh <fredrik@pythonware.com> parents: 38182 diff changeset	5577 Unicode string S[start:end]. Optional arguments start and end are\n\
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5578 interpreted as in slice notation.");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5579
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5580 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5581 unicode_count(PyUnicodeObject self, PyObject args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5582 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5583 PyUnicodeObject *substring;
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	5584 Py_ssize_t start = 0;
37662 0678ce9fb6b2 Change more ints to Py_ssize_t. Martin v. Löwis <martin@v.loewis.de> parents: 37661 diff changeset	5585 Py_ssize_t end = PY_SSIZE_T_MAX;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5586 PyObject *result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5587
12629 3efeaa7521e4 Trent Mick: Guido van Rossum <guido@python.org> parents: 12582 diff changeset	5588 if (!PyArg_ParseTuple(args, "O\|O&O&:count", &substring,
3efeaa7521e4 Trent Mick: Guido van Rossum <guido@python.org> parents: 12582 diff changeset	5589 _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end))
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5590 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5591
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5592 substring = (PyUnicodeObject *)PyUnicode_FromObject(
38400 33c328896f52 needforspeed: stringlib refactoring, continued. added count and Fredrik Lundh <fredrik@pythonware.com> parents: 38391 diff changeset	5593 (PyObject *)substring);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5594 if (substring == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5595 return NULL;
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	5596
38402 6b03a47eb54d needforspeed: use a macro to fix slice indexes Fredrik Lundh <fredrik@pythonware.com> parents: 38401 diff changeset	5597 FIX_START_END(self);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5598
38400 33c328896f52 needforspeed: stringlib refactoring, continued. added count and Fredrik Lundh <fredrik@pythonware.com> parents: 38391 diff changeset	5599 result = PyInt_FromSsize_t(
33c328896f52 needforspeed: stringlib refactoring, continued. added count and Fredrik Lundh <fredrik@pythonware.com> parents: 38391 diff changeset	5600 stringlib_count(self->str + start, end - start,
33c328896f52 needforspeed: stringlib refactoring, continued. added count and Fredrik Lundh <fredrik@pythonware.com> parents: 38391 diff changeset	5601 substring->str, substring->length)
33c328896f52 needforspeed: stringlib refactoring, continued. added count and Fredrik Lundh <fredrik@pythonware.com> parents: 38391 diff changeset	5602 );
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5603
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5604 Py_DECREF(substring);
38400 33c328896f52 needforspeed: stringlib refactoring, continued. added count and Fredrik Lundh <fredrik@pythonware.com> parents: 38391 diff changeset	5605
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5606 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5607 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5608
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5609 PyDoc_STRVAR(encode__doc__,
32551 f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	5610 "S.encode([encoding[,errors]]) -> string or unicode\n\
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5611 \n\
32551 f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	5612 Encodes S using the codec registered for encoding. encoding defaults\n\
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	5613 to the default encoding. errors may be given to set a different error\n\
12645 c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	5614 handling scheme. Default is 'strict' meaning that encoding errors raise\n\
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	5615 a UnicodeEncodeError. Other possible values are 'ignore', 'replace' and\n\
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	5616 'xmlcharrefreplace' as well as any other name registered with\n\
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	5617 codecs.register_error that can handle UnicodeEncodeErrors.");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5618
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5619 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5620 unicode_encode(PyUnicodeObject self, PyObject args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5621 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5622 char *encoding = NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5623 char *errors = NULL;
32551 f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	5624 PyObject *v;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	5625
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5626 if (!PyArg_ParseTuple(args, "\|ss:encode", &encoding, &errors))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5627 return NULL;
32551 f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	5628 v = PyUnicode_AsEncodedObject((PyObject *)self, encoding, errors);
32552 3706a3704aea .encode()/.decode() patch part 2. Marc-André Lemburg <mal@egenix.com> parents: 32551 diff changeset	5629 if (v == NULL)
3706a3704aea .encode()/.decode() patch part 2. Marc-André Lemburg <mal@egenix.com> parents: 32551 diff changeset	5630 goto onError;
32551 f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	5631 if (!PyString_Check(v) && !PyUnicode_Check(v)) {
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	5632 PyErr_Format(PyExc_TypeError,
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	5633 "encoder did not return a string/unicode object "
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	5634 "(type=%.400s)",
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	5635 v->ob_type->tp_name);
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	5636 Py_DECREF(v);
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	5637 return NULL;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	5638 }
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	5639 return v;
32552 3706a3704aea .encode()/.decode() patch part 2. Marc-André Lemburg <mal@egenix.com> parents: 32551 diff changeset	5640
3706a3704aea .encode()/.decode() patch part 2. Marc-André Lemburg <mal@egenix.com> parents: 32551 diff changeset	5641 onError:
3706a3704aea .encode()/.decode() patch part 2. Marc-André Lemburg <mal@egenix.com> parents: 32551 diff changeset	5642 return NULL;
32551 f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	5643 }
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	5644
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	5645 PyDoc_STRVAR(decode__doc__,
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	5646 "S.decode([encoding[,errors]]) -> string or unicode\n\
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	5647 \n\
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	5648 Decodes S using the codec registered for encoding. encoding defaults\n\
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	5649 to the default encoding. errors may be given to set a different error\n\
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	5650 handling scheme. Default is 'strict' meaning that encoding errors raise\n\
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	5651 a UnicodeDecodeError. Other possible values are 'ignore' and 'replace'\n\
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	5652 as well as any other name registerd with codecs.register_error that is\n\
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	5653 able to handle UnicodeDecodeErrors.");
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	5654
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	5655 static PyObject *
32571 bec0c7618625 Fix a copy&paste typo. Marc-André Lemburg <mal@egenix.com> parents: 32552 diff changeset	5656 unicode_decode(PyUnicodeObject self, PyObject args)
32551 f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	5657 {
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	5658 char *encoding = NULL;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	5659 char *errors = NULL;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	5660 PyObject *v;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	5661
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	5662 if (!PyArg_ParseTuple(args, "\|ss:decode", &encoding, &errors))
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	5663 return NULL;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	5664 v = PyUnicode_AsDecodedObject((PyObject *)self, encoding, errors);
32552 3706a3704aea .encode()/.decode() patch part 2. Marc-André Lemburg <mal@egenix.com> parents: 32551 diff changeset	5665 if (v == NULL)
3706a3704aea .encode()/.decode() patch part 2. Marc-André Lemburg <mal@egenix.com> parents: 32551 diff changeset	5666 goto onError;
32551 f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	5667 if (!PyString_Check(v) && !PyUnicode_Check(v)) {
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	5668 PyErr_Format(PyExc_TypeError,
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	5669 "decoder did not return a string/unicode object "
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	5670 "(type=%.400s)",
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	5671 v->ob_type->tp_name);
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	5672 Py_DECREF(v);
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	5673 return NULL;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	5674 }
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	5675 return v;
32552 3706a3704aea .encode()/.decode() patch part 2. Marc-André Lemburg <mal@egenix.com> parents: 32551 diff changeset	5676
3706a3704aea .encode()/.decode() patch part 2. Marc-André Lemburg <mal@egenix.com> parents: 32551 diff changeset	5677 onError:
3706a3704aea .encode()/.decode() patch part 2. Marc-André Lemburg <mal@egenix.com> parents: 32551 diff changeset	5678 return NULL;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5679 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5680
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5681 PyDoc_STRVAR(expandtabs__doc__,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5682 "S.expandtabs([tabsize]) -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5683 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5684 Return a copy of S where all tab characters are expanded using spaces.\n\
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5685 If tabsize is not given, a tab size of 8 characters is assumed.");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5686
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5687 static PyObject*
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5688 unicode_expandtabs(PyUnicodeObject self, PyObject args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5689 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5690 Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5691 Py_UNICODE *p;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5692 Py_UNICODE *q;
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	5693 Py_ssize_t i, j;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5694 PyUnicodeObject *u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5695 int tabsize = 8;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5696
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5697 if (!PyArg_ParseTuple(args, "\|i:expandtabs", &tabsize))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5698 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5699
13620 5b727ab37092 Spelling fixes supplied by Rob W. W. Hooft. All these are fixes in either Thomas Wouters <thomas@python.org> parents: 13576 diff changeset	5700 /* First pass: determine size of output string */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5701 i = j = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5702 e = self->str + self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5703 for (p = self->str; p < e; p++)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5704 if (*p == '\t') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5705 if (tabsize > 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5706 j += tabsize - (j % tabsize);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5707 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5708 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5709 j++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5710 if (p == '\n' \|\| p == '\r') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5711 i += j;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5712 j = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5713 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5714 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5715
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5716 /* Second pass: create output string and fill it */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5717 u = _PyUnicode_New(i + j);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5718 if (!u)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5719 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5720
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5721 j = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5722 q = u->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5723
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5724 for (p = self->str; p < e; p++)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5725 if (*p == '\t') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5726 if (tabsize > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5727 i = tabsize - (j % tabsize);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5728 j += i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5729 while (i--)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5730 *q++ = ' ';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5731 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5732 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5733 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5734 j++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5735 q++ = p;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5736 if (p == '\n' \|\| p == '\r')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5737 j = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5738 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5739
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5740 return (PyObject*) u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5741 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5742
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5743 PyDoc_STRVAR(find__doc__,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5744 "S.find(sub [,start [,end]]) -> int\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5745 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5746 Return the lowest index in S where substring sub is found,\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5747 such that sub is contained within s[start,end]. Optional\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5748 arguments start and end are interpreted as in slice notation.\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5749 \n\
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5750 Return -1 on failure.");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5751
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5752 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5753 unicode_find(PyUnicodeObject self, PyObject args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5754 {
38424 3d81faf47af7 needforspeed: more stringlib refactoring Fredrik Lundh <fredrik@pythonware.com> parents: 38422 diff changeset	5755 PyObject *substring;
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	5756 Py_ssize_t start = 0;
37662 0678ce9fb6b2 Change more ints to Py_ssize_t. Martin v. Löwis <martin@v.loewis.de> parents: 37661 diff changeset	5757 Py_ssize_t end = PY_SSIZE_T_MAX;
38424 3d81faf47af7 needforspeed: more stringlib refactoring Fredrik Lundh <fredrik@pythonware.com> parents: 38422 diff changeset	5758 Py_ssize_t result;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5759
12629 3efeaa7521e4 Trent Mick: Guido van Rossum <guido@python.org> parents: 12582 diff changeset	5760 if (!PyArg_ParseTuple(args, "O\|O&O&:find", &substring,
3efeaa7521e4 Trent Mick: Guido van Rossum <guido@python.org> parents: 12582 diff changeset	5761 _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end))
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5762 return NULL;
38424 3d81faf47af7 needforspeed: more stringlib refactoring Fredrik Lundh <fredrik@pythonware.com> parents: 38422 diff changeset	5763 substring = PyUnicode_FromObject(substring);
3d81faf47af7 needforspeed: more stringlib refactoring Fredrik Lundh <fredrik@pythonware.com> parents: 38422 diff changeset	5764 if (!substring)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5765 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5766
38441 72327f8f4a79 needforspeed: stringlib refactoring: changed find_obj to find_slice, Fredrik Lundh <fredrik@pythonware.com> parents: 38440 diff changeset	5767 result = stringlib_find_slice(
72327f8f4a79 needforspeed: stringlib refactoring: changed find_obj to find_slice, Fredrik Lundh <fredrik@pythonware.com> parents: 38440 diff changeset	5768 PyUnicode_AS_UNICODE(self), PyUnicode_GET_SIZE(self),
72327f8f4a79 needforspeed: stringlib refactoring: changed find_obj to find_slice, Fredrik Lundh <fredrik@pythonware.com> parents: 38440 diff changeset	5769 PyUnicode_AS_UNICODE(substring), PyUnicode_GET_SIZE(substring),
72327f8f4a79 needforspeed: stringlib refactoring: changed find_obj to find_slice, Fredrik Lundh <fredrik@pythonware.com> parents: 38440 diff changeset	5770 start, end
72327f8f4a79 needforspeed: stringlib refactoring: changed find_obj to find_slice, Fredrik Lundh <fredrik@pythonware.com> parents: 38440 diff changeset	5771 );
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5772
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5773 Py_DECREF(substring);
38424 3d81faf47af7 needforspeed: more stringlib refactoring Fredrik Lundh <fredrik@pythonware.com> parents: 38422 diff changeset	5774
3d81faf47af7 needforspeed: more stringlib refactoring Fredrik Lundh <fredrik@pythonware.com> parents: 38422 diff changeset	5775 return PyInt_FromSsize_t(result);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5776 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5777
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5778 static PyObject *
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	5779 unicode_getitem(PyUnicodeObject *self, Py_ssize_t index)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5780 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5781 if (index < 0 \|\| index >= self->length) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5782 PyErr_SetString(PyExc_IndexError, "string index out of range");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5783 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5784 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5785
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5786 return (PyObject*) PyUnicode_FromUnicode(&self->str[index], 1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5787 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5788
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5789 static long
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5790 unicode_hash(PyUnicodeObject *self)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5791 {
13522 acc4638f37d5 - changed hash calculation for unicode strings. the new Fredrik Lundh <fredrik@pythonware.com> parents: 13411 diff changeset	5792 /* Since Unicode objects compare equal to their ASCII string
acc4638f37d5 - changed hash calculation for unicode strings. the new Fredrik Lundh <fredrik@pythonware.com> parents: 13411 diff changeset	5793 counterparts, they should use the individual character values
acc4638f37d5 - changed hash calculation for unicode strings. the new Fredrik Lundh <fredrik@pythonware.com> parents: 13411 diff changeset	5794 as basis for their hash value. This is needed to assure that
acc4638f37d5 - changed hash calculation for unicode strings. the new Fredrik Lundh <fredrik@pythonware.com> parents: 13411 diff changeset	5795 strings and Unicode objects behave in the same way as
acc4638f37d5 - changed hash calculation for unicode strings. the new Fredrik Lundh <fredrik@pythonware.com> parents: 13411 diff changeset	5796 dictionary keys. */
acc4638f37d5 - changed hash calculation for unicode strings. the new Fredrik Lundh <fredrik@pythonware.com> parents: 13411 diff changeset	5797
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	5798 register Py_ssize_t len;
13522 acc4638f37d5 - changed hash calculation for unicode strings. the new Fredrik Lundh <fredrik@pythonware.com> parents: 13411 diff changeset	5799 register Py_UNICODE *p;
acc4638f37d5 - changed hash calculation for unicode strings. the new Fredrik Lundh <fredrik@pythonware.com> parents: 13411 diff changeset	5800 register long x;
acc4638f37d5 - changed hash calculation for unicode strings. the new Fredrik Lundh <fredrik@pythonware.com> parents: 13411 diff changeset	5801
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5802 if (self->hash != -1)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5803 return self->hash;
13522 acc4638f37d5 - changed hash calculation for unicode strings. the new Fredrik Lundh <fredrik@pythonware.com> parents: 13411 diff changeset	5804 len = PyUnicode_GET_SIZE(self);
acc4638f37d5 - changed hash calculation for unicode strings. the new Fredrik Lundh <fredrik@pythonware.com> parents: 13411 diff changeset	5805 p = PyUnicode_AS_UNICODE(self);
acc4638f37d5 - changed hash calculation for unicode strings. the new Fredrik Lundh <fredrik@pythonware.com> parents: 13411 diff changeset	5806 x = *p << 7;
acc4638f37d5 - changed hash calculation for unicode strings. the new Fredrik Lundh <fredrik@pythonware.com> parents: 13411 diff changeset	5807 while (--len >= 0)
acc4638f37d5 - changed hash calculation for unicode strings. the new Fredrik Lundh <fredrik@pythonware.com> parents: 13411 diff changeset	5808 x = (1000003x) ^ p++;
acc4638f37d5 - changed hash calculation for unicode strings. the new Fredrik Lundh <fredrik@pythonware.com> parents: 13411 diff changeset	5809 x ^= PyUnicode_GET_SIZE(self);
acc4638f37d5 - changed hash calculation for unicode strings. the new Fredrik Lundh <fredrik@pythonware.com> parents: 13411 diff changeset	5810 if (x == -1)
acc4638f37d5 - changed hash calculation for unicode strings. the new Fredrik Lundh <fredrik@pythonware.com> parents: 13411 diff changeset	5811 x = -2;
acc4638f37d5 - changed hash calculation for unicode strings. the new Fredrik Lundh <fredrik@pythonware.com> parents: 13411 diff changeset	5812 self->hash = x;
acc4638f37d5 - changed hash calculation for unicode strings. the new Fredrik Lundh <fredrik@pythonware.com> parents: 13411 diff changeset	5813 return x;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5814 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5815
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5816 PyDoc_STRVAR(index__doc__,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5817 "S.index(sub [,start [,end]]) -> int\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5818 \n\
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5819 Like S.find() but raise ValueError when the substring is not found.");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5820
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5821 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5822 unicode_index(PyUnicodeObject self, PyObject args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5823 {
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	5824 Py_ssize_t result;
38424 3d81faf47af7 needforspeed: more stringlib refactoring Fredrik Lundh <fredrik@pythonware.com> parents: 38422 diff changeset	5825 PyObject *substring;
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	5826 Py_ssize_t start = 0;
37662 0678ce9fb6b2 Change more ints to Py_ssize_t. Martin v. Löwis <martin@v.loewis.de> parents: 37661 diff changeset	5827 Py_ssize_t end = PY_SSIZE_T_MAX;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5828
12629 3efeaa7521e4 Trent Mick: Guido van Rossum <guido@python.org> parents: 12582 diff changeset	5829 if (!PyArg_ParseTuple(args, "O\|O&O&:index", &substring,
3efeaa7521e4 Trent Mick: Guido van Rossum <guido@python.org> parents: 12582 diff changeset	5830 _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end))
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5831 return NULL;
38424 3d81faf47af7 needforspeed: more stringlib refactoring Fredrik Lundh <fredrik@pythonware.com> parents: 38422 diff changeset	5832 substring = PyUnicode_FromObject(substring);
3d81faf47af7 needforspeed: more stringlib refactoring Fredrik Lundh <fredrik@pythonware.com> parents: 38422 diff changeset	5833 if (!substring)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5834 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5835
38441 72327f8f4a79 needforspeed: stringlib refactoring: changed find_obj to find_slice, Fredrik Lundh <fredrik@pythonware.com> parents: 38440 diff changeset	5836 result = stringlib_find_slice(
72327f8f4a79 needforspeed: stringlib refactoring: changed find_obj to find_slice, Fredrik Lundh <fredrik@pythonware.com> parents: 38440 diff changeset	5837 PyUnicode_AS_UNICODE(self), PyUnicode_GET_SIZE(self),
72327f8f4a79 needforspeed: stringlib refactoring: changed find_obj to find_slice, Fredrik Lundh <fredrik@pythonware.com> parents: 38440 diff changeset	5838 PyUnicode_AS_UNICODE(substring), PyUnicode_GET_SIZE(substring),
72327f8f4a79 needforspeed: stringlib refactoring: changed find_obj to find_slice, Fredrik Lundh <fredrik@pythonware.com> parents: 38440 diff changeset	5839 start, end
72327f8f4a79 needforspeed: stringlib refactoring: changed find_obj to find_slice, Fredrik Lundh <fredrik@pythonware.com> parents: 38440 diff changeset	5840 );
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5841
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5842 Py_DECREF(substring);
38424 3d81faf47af7 needforspeed: more stringlib refactoring Fredrik Lundh <fredrik@pythonware.com> parents: 38422 diff changeset	5843
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5844 if (result < 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5845 PyErr_SetString(PyExc_ValueError, "substring not found");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5846 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5847 }
38424 3d81faf47af7 needforspeed: more stringlib refactoring Fredrik Lundh <fredrik@pythonware.com> parents: 38422 diff changeset	5848
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	5849 return PyInt_FromSsize_t(result);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5850 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5851
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5852 PyDoc_STRVAR(islower__doc__,
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5853 "S.islower() -> bool\n\
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5854 \n\
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5855 Return True if all cased characters in S are lowercase and there is\n\
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5856 at least one cased character in S, False otherwise.");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5857
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5858 static PyObject*
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	5859 unicode_islower(PyUnicodeObject *self)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5860 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5861 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5862 register const Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5863 int cased;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5864
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5865 /* Shortcut for single character strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5866 if (PyUnicode_GET_SIZE(self) == 1)
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5867 return PyBool_FromLong(Py_UNICODE_ISLOWER(*p));
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5868
12924 2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	5869 /* Special case for empty strings */
36203 44cfca6136c6 Stop maintaining the buildno file. Martin v. Löwis <martin@v.loewis.de> parents: 36056 diff changeset	5870 if (PyUnicode_GET_SIZE(self) == 0)
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5871 return PyBool_FromLong(0);
12924 2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	5872
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5873 e = p + PyUnicode_GET_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5874 cased = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5875 for (; p < e; p++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5876 register const Py_UNICODE ch = *p;
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	5877
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5878 if (Py_UNICODE_ISUPPER(ch) \|\| Py_UNICODE_ISTITLE(ch))
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5879 return PyBool_FromLong(0);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5880 else if (!cased && Py_UNICODE_ISLOWER(ch))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5881 cased = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5882 }
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5883 return PyBool_FromLong(cased);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5884 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5885
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5886 PyDoc_STRVAR(isupper__doc__,
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5887 "S.isupper() -> bool\n\
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5888 \n\
30707 9e21e56bf5f3 Patch #825679: Clarify semantics of .isfoo on empty strings. Martin v. Löwis <martin@v.loewis.de> parents: 30653 diff changeset	5889 Return True if all cased characters in S are uppercase and there is\n\
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5890 at least one cased character in S, False otherwise.");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5891
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5892 static PyObject*
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	5893 unicode_isupper(PyUnicodeObject *self)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5894 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5895 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5896 register const Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5897 int cased;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5898
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5899 /* Shortcut for single character strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5900 if (PyUnicode_GET_SIZE(self) == 1)
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5901 return PyBool_FromLong(Py_UNICODE_ISUPPER(*p) != 0);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5902
12924 2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	5903 /* Special case for empty strings */
36203 44cfca6136c6 Stop maintaining the buildno file. Martin v. Löwis <martin@v.loewis.de> parents: 36056 diff changeset	5904 if (PyUnicode_GET_SIZE(self) == 0)
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5905 return PyBool_FromLong(0);
12924 2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	5906
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5907 e = p + PyUnicode_GET_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5908 cased = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5909 for (; p < e; p++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5910 register const Py_UNICODE ch = *p;
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	5911
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5912 if (Py_UNICODE_ISLOWER(ch) \|\| Py_UNICODE_ISTITLE(ch))
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5913 return PyBool_FromLong(0);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5914 else if (!cased && Py_UNICODE_ISUPPER(ch))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5915 cased = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5916 }
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5917 return PyBool_FromLong(cased);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5918 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5919
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5920 PyDoc_STRVAR(istitle__doc__,
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5921 "S.istitle() -> bool\n\
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5922 \n\
30707 9e21e56bf5f3 Patch #825679: Clarify semantics of .isfoo on empty strings. Martin v. Löwis <martin@v.loewis.de> parents: 30653 diff changeset	5923 Return True if S is a titlecased string and there is at least one\n\
9e21e56bf5f3 Patch #825679: Clarify semantics of .isfoo on empty strings. Martin v. Löwis <martin@v.loewis.de> parents: 30653 diff changeset	5924 character in S, i.e. upper- and titlecase characters may only\n\
9e21e56bf5f3 Patch #825679: Clarify semantics of .isfoo on empty strings. Martin v. Löwis <martin@v.loewis.de> parents: 30653 diff changeset	5925 follow uncased characters and lowercase characters only cased ones.\n\
9e21e56bf5f3 Patch #825679: Clarify semantics of .isfoo on empty strings. Martin v. Löwis <martin@v.loewis.de> parents: 30653 diff changeset	5926 Return False otherwise.");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5927
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5928 static PyObject*
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	5929 unicode_istitle(PyUnicodeObject *self)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5930 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5931 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5932 register const Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5933 int cased, previous_is_cased;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5934
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5935 /* Shortcut for single character strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5936 if (PyUnicode_GET_SIZE(self) == 1)
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5937 return PyBool_FromLong((Py_UNICODE_ISTITLE(*p) != 0) \|\|
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5938 (Py_UNICODE_ISUPPER(*p) != 0));
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5939
12924 2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	5940 /* Special case for empty strings */
36203 44cfca6136c6 Stop maintaining the buildno file. Martin v. Löwis <martin@v.loewis.de> parents: 36056 diff changeset	5941 if (PyUnicode_GET_SIZE(self) == 0)
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5942 return PyBool_FromLong(0);
12924 2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	5943
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5944 e = p + PyUnicode_GET_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5945 cased = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5946 previous_is_cased = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5947 for (; p < e; p++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5948 register const Py_UNICODE ch = *p;
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	5949
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5950 if (Py_UNICODE_ISUPPER(ch) \|\| Py_UNICODE_ISTITLE(ch)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5951 if (previous_is_cased)
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5952 return PyBool_FromLong(0);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5953 previous_is_cased = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5954 cased = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5955 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5956 else if (Py_UNICODE_ISLOWER(ch)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5957 if (!previous_is_cased)
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5958 return PyBool_FromLong(0);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5959 previous_is_cased = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5960 cased = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5961 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5962 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5963 previous_is_cased = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5964 }
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5965 return PyBool_FromLong(cased);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5966 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5967
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5968 PyDoc_STRVAR(isspace__doc__,
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5969 "S.isspace() -> bool\n\
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5970 \n\
30707 9e21e56bf5f3 Patch #825679: Clarify semantics of .isfoo on empty strings. Martin v. Löwis <martin@v.loewis.de> parents: 30653 diff changeset	5971 Return True if all characters in S are whitespace\n\
9e21e56bf5f3 Patch #825679: Clarify semantics of .isfoo on empty strings. Martin v. Löwis <martin@v.loewis.de> parents: 30653 diff changeset	5972 and there is at least one character in S, False otherwise.");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5973
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5974 static PyObject*
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	5975 unicode_isspace(PyUnicodeObject *self)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5976 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5977 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5978 register const Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5979
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5980 /* Shortcut for single character strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5981 if (PyUnicode_GET_SIZE(self) == 1 &&
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5982 Py_UNICODE_ISSPACE(*p))
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5983 return PyBool_FromLong(1);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5984
12924 2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	5985 /* Special case for empty strings */
36203 44cfca6136c6 Stop maintaining the buildno file. Martin v. Löwis <martin@v.loewis.de> parents: 36056 diff changeset	5986 if (PyUnicode_GET_SIZE(self) == 0)
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5987 return PyBool_FromLong(0);
12924 2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	5988
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5989 e = p + PyUnicode_GET_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5990 for (; p < e; p++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5991 if (!Py_UNICODE_ISSPACE(*p))
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5992 return PyBool_FromLong(0);
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5993 }
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5994 return PyBool_FromLong(1);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5995 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5996
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5997 PyDoc_STRVAR(isalpha__doc__,
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5998 "S.isalpha() -> bool\n\
13356 9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	5999 \n\
30707 9e21e56bf5f3 Patch #825679: Clarify semantics of .isfoo on empty strings. Martin v. Löwis <martin@v.loewis.de> parents: 30653 diff changeset	6000 Return True if all characters in S are alphabetic\n\
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	6001 and there is at least one character in S, False otherwise.");
13356 9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	6002
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	6003 static PyObject*
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	6004 unicode_isalpha(PyUnicodeObject *self)
13356 9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	6005 {
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	6006 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	6007 register const Py_UNICODE *e;
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	6008
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	6009 /* Shortcut for single character strings */
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	6010 if (PyUnicode_GET_SIZE(self) == 1 &&
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	6011 Py_UNICODE_ISALPHA(*p))
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	6012 return PyBool_FromLong(1);
13356 9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	6013
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	6014 /* Special case for empty strings */
36203 44cfca6136c6 Stop maintaining the buildno file. Martin v. Löwis <martin@v.loewis.de> parents: 36056 diff changeset	6015 if (PyUnicode_GET_SIZE(self) == 0)
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	6016 return PyBool_FromLong(0);
13356 9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	6017
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	6018 e = p + PyUnicode_GET_SIZE(self);
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	6019 for (; p < e; p++) {
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	6020 if (!Py_UNICODE_ISALPHA(*p))
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	6021 return PyBool_FromLong(0);
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	6022 }
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	6023 return PyBool_FromLong(1);
13356 9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	6024 }
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	6025
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	6026 PyDoc_STRVAR(isalnum__doc__,
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	6027 "S.isalnum() -> bool\n\
13356 9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	6028 \n\
30707 9e21e56bf5f3 Patch #825679: Clarify semantics of .isfoo on empty strings. Martin v. Löwis <martin@v.loewis.de> parents: 30653 diff changeset	6029 Return True if all characters in S are alphanumeric\n\
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	6030 and there is at least one character in S, False otherwise.");
13356 9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	6031
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	6032 static PyObject*
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	6033 unicode_isalnum(PyUnicodeObject *self)
13356 9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	6034 {
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	6035 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	6036 register const Py_UNICODE *e;
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	6037
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	6038 /* Shortcut for single character strings */
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	6039 if (PyUnicode_GET_SIZE(self) == 1 &&
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	6040 Py_UNICODE_ISALNUM(*p))
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	6041 return PyBool_FromLong(1);
13356 9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	6042
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	6043 /* Special case for empty strings */
36203 44cfca6136c6 Stop maintaining the buildno file. Martin v. Löwis <martin@v.loewis.de> parents: 36056 diff changeset	6044 if (PyUnicode_GET_SIZE(self) == 0)
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	6045 return PyBool_FromLong(0);
13356 9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	6046
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	6047 e = p + PyUnicode_GET_SIZE(self);
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	6048 for (; p < e; p++) {
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	6049 if (!Py_UNICODE_ISALNUM(*p))
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	6050 return PyBool_FromLong(0);
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	6051 }
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	6052 return PyBool_FromLong(1);
13356 9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	6053 }
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	6054
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	6055 PyDoc_STRVAR(isdecimal__doc__,
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	6056 "S.isdecimal() -> bool\n\
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6057 \n\
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	6058 Return True if there are only decimal characters in S,\n\
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	6059 False otherwise.");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6060
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6061 static PyObject*
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	6062 unicode_isdecimal(PyUnicodeObject *self)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6063 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6064 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6065 register const Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6066
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6067 /* Shortcut for single character strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6068 if (PyUnicode_GET_SIZE(self) == 1 &&
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6069 Py_UNICODE_ISDECIMAL(*p))
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	6070 return PyBool_FromLong(1);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6071
12924 2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	6072 /* Special case for empty strings */
36203 44cfca6136c6 Stop maintaining the buildno file. Martin v. Löwis <martin@v.loewis.de> parents: 36056 diff changeset	6073 if (PyUnicode_GET_SIZE(self) == 0)
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	6074 return PyBool_FromLong(0);
12924 2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	6075
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6076 e = p + PyUnicode_GET_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6077 for (; p < e; p++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6078 if (!Py_UNICODE_ISDECIMAL(*p))
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	6079 return PyBool_FromLong(0);
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	6080 }
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	6081 return PyBool_FromLong(1);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6082 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6083
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	6084 PyDoc_STRVAR(isdigit__doc__,
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	6085 "S.isdigit() -> bool\n\
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6086 \n\
30707 9e21e56bf5f3 Patch #825679: Clarify semantics of .isfoo on empty strings. Martin v. Löwis <martin@v.loewis.de> parents: 30653 diff changeset	6087 Return True if all characters in S are digits\n\
9e21e56bf5f3 Patch #825679: Clarify semantics of .isfoo on empty strings. Martin v. Löwis <martin@v.loewis.de> parents: 30653 diff changeset	6088 and there is at least one character in S, False otherwise.");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6089
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6090 static PyObject*
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	6091 unicode_isdigit(PyUnicodeObject *self)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6092 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6093 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6094 register const Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6095
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6096 /* Shortcut for single character strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6097 if (PyUnicode_GET_SIZE(self) == 1 &&
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6098 Py_UNICODE_ISDIGIT(*p))
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	6099 return PyBool_FromLong(1);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6100
12924 2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	6101 /* Special case for empty strings */
36203 44cfca6136c6 Stop maintaining the buildno file. Martin v. Löwis <martin@v.loewis.de> parents: 36056 diff changeset	6102 if (PyUnicode_GET_SIZE(self) == 0)
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	6103 return PyBool_FromLong(0);
12924 2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	6104
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6105 e = p + PyUnicode_GET_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6106 for (; p < e; p++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6107 if (!Py_UNICODE_ISDIGIT(*p))
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	6108 return PyBool_FromLong(0);
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	6109 }
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	6110 return PyBool_FromLong(1);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6111 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6112
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	6113 PyDoc_STRVAR(isnumeric__doc__,
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	6114 "S.isnumeric() -> bool\n\
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6115 \n\
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	6116 Return True if there are only numeric characters in S,\n\
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	6117 False otherwise.");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6118
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6119 static PyObject*
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	6120 unicode_isnumeric(PyUnicodeObject *self)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6121 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6122 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6123 register const Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6124
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6125 /* Shortcut for single character strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6126 if (PyUnicode_GET_SIZE(self) == 1 &&
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6127 Py_UNICODE_ISNUMERIC(*p))
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	6128 return PyBool_FromLong(1);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6129
12924 2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	6130 /* Special case for empty strings */
36203 44cfca6136c6 Stop maintaining the buildno file. Martin v. Löwis <martin@v.loewis.de> parents: 36056 diff changeset	6131 if (PyUnicode_GET_SIZE(self) == 0)
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	6132 return PyBool_FromLong(0);
12924 2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	6133
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6134 e = p + PyUnicode_GET_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6135 for (; p < e; p++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6136 if (!Py_UNICODE_ISNUMERIC(*p))
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	6137 return PyBool_FromLong(0);
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	6138 }
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	6139 return PyBool_FromLong(1);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6140 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6141
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	6142 PyDoc_STRVAR(join__doc__,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6143 "S.join(sequence) -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6144 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6145 Return a string which is the concatenation of the strings in the\n\
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	6146 sequence. The separator between elements is S.");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6147
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6148 static PyObject*
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	6149 unicode_join(PyObject self, PyObject data)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6150 {
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	6151 return PyUnicode_Join(self, data);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6152 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6153
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	6154 static Py_ssize_t
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6155 unicode_length(PyUnicodeObject *self)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6156 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6157 return self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6158 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6159
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	6160 PyDoc_STRVAR(ljust__doc__,
32197 3a84e23b042f - SF #962502: Add two more methods for unicode type; width() and Hye-Shik Chang <hyeshik@gmail.com> parents: 31873 diff changeset	6161 "S.ljust(width[, fillchar]) -> int\n\
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6162 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6163 Return S left justified in a Unicode string of length width. Padding is\n\
31021 21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods. Raymond Hettinger <python@rcn.com> parents: 30765 diff changeset	6164 done using the specified fill character (default is a space).");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6165
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6166 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6167 unicode_ljust(PyUnicodeObject self, PyObject args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6168 {
37662 0678ce9fb6b2 Change more ints to Py_ssize_t. Martin v. Löwis <martin@v.loewis.de> parents: 37661 diff changeset	6169 Py_ssize_t width;
31021 21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods. Raymond Hettinger <python@rcn.com> parents: 30765 diff changeset	6170 Py_UNICODE fillchar = ' ';
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods. Raymond Hettinger <python@rcn.com> parents: 30765 diff changeset	6171
37662 0678ce9fb6b2 Change more ints to Py_ssize_t. Martin v. Löwis <martin@v.loewis.de> parents: 37661 diff changeset	6172 if (!PyArg_ParseTuple(args, "n\|O&:ljust", &width, convert_uc, &fillchar))
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6173 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6174
19885 72e6eb653312 More on bug 460020: disable many optimizations of unicode subclasses. Tim Peters <tim.peters@gmail.com> parents: 19849 diff changeset	6175 if (self->length >= width && PyUnicode_CheckExact(self)) {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6176 Py_INCREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6177 return (PyObject*) self;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6178 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6179
31021 21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods. Raymond Hettinger <python@rcn.com> parents: 30765 diff changeset	6180 return (PyObject*) pad(self, 0, width - self->length, fillchar);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6181 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6182
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	6183 PyDoc_STRVAR(lower__doc__,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6184 "S.lower() -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6185 \n\
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	6186 Return a copy of the string S converted to lowercase.");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6187
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6188 static PyObject*
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	6189 unicode_lower(PyUnicodeObject *self)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6190 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6191 return fixup(self, fixlower);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6192 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6193
23225 e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6194 #define LEFTSTRIP 0
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6195 #define RIGHTSTRIP 1
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6196 #define BOTHSTRIP 2
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6197
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6198 /* Arrays indexed by above */
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6199 static const char *stripformat[] = {"\|O:lstrip", "\|O:rstrip", "\|O:strip"};
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6200
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6201 #define STRIPNAME(i) (stripformat[i]+3)
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6202
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6203 /* externally visible for str.strip(unicode) */
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6204 PyObject *
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6205 _PyUnicode_XStrip(PyUnicodeObject self, int striptype, PyObject sepobj)
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6206 {
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6207 Py_UNICODE *s = PyUnicode_AS_UNICODE(self);
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	6208 Py_ssize_t len = PyUnicode_GET_SIZE(self);
23225 e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6209 Py_UNICODE *sep = PyUnicode_AS_UNICODE(sepobj);
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	6210 Py_ssize_t seplen = PyUnicode_GET_SIZE(sepobj);
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	6211 Py_ssize_t i, j;
23225 e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6212
38257 af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	6213 BLOOM_MASK sepmask = make_bloom_mask(sep, seplen);
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	6214
23225 e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6215 i = 0;
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6216 if (striptype != RIGHTSTRIP) {
38257 af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	6217 while (i < len && BLOOM_MEMBER(sepmask, s[i], sep, seplen)) {
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	6218 i++;
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	6219 }
23225 e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6220 }
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6221
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6222 j = len;
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6223 if (striptype != LEFTSTRIP) {
38257 af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	6224 do {
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	6225 j--;
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	6226 } while (j >= i && BLOOM_MEMBER(sepmask, s[j], sep, seplen));
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	6227 j++;
23225 e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6228 }
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6229
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6230 if (i == 0 && j == len && PyUnicode_CheckExact(self)) {
38257 af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	6231 Py_INCREF(self);
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	6232 return (PyObject*)self;
23225 e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6233 }
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6234 else
38257 af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	6235 return PyUnicode_FromUnicode(s+i, j-i);
23225 e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6236 }
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6237
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6238
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6239 static PyObject *
23225 e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6240 do_strip(PyUnicodeObject *self, int striptype)
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6241 {
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6242 Py_UNICODE *s = PyUnicode_AS_UNICODE(self);
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	6243 Py_ssize_t len = PyUnicode_GET_SIZE(self), i, j;
23225 e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6244
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6245 i = 0;
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6246 if (striptype != RIGHTSTRIP) {
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6247 while (i < len && Py_UNICODE_ISSPACE(s[i])) {
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6248 i++;
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6249 }
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6250 }
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6251
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6252 j = len;
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6253 if (striptype != LEFTSTRIP) {
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6254 do {
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6255 j--;
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6256 } while (j >= i && Py_UNICODE_ISSPACE(s[j]));
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6257 j++;
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6258 }
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6259
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6260 if (i == 0 && j == len && PyUnicode_CheckExact(self)) {
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6261 Py_INCREF(self);
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6262 return (PyObject*)self;
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6263 }
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6264 else
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6265 return PyUnicode_FromUnicode(s+i, j-i);
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6266 }
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6267
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6268
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6269 static PyObject *
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6270 do_argstrip(PyUnicodeObject self, int striptype, PyObject args)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6271 {
23225 e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6272 PyObject *sep = NULL;
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6273
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6274 if (!PyArg_ParseTuple(args, (char *)stripformat[striptype], &sep))
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6275 return NULL;
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6276
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6277 if (sep != NULL && sep != Py_None) {
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6278 if (PyUnicode_Check(sep))
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6279 return _PyUnicode_XStrip(self, striptype, sep);
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6280 else if (PyString_Check(sep)) {
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6281 PyObject *res;
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6282 sep = PyUnicode_FromObject(sep);
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6283 if (sep==NULL)
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6284 return NULL;
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6285 res = _PyUnicode_XStrip(self, striptype, sep);
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6286 Py_DECREF(sep);
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6287 return res;
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6288 }
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6289 else {
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6290 PyErr_Format(PyExc_TypeError,
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6291 "%s arg must be None, unicode or str",
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6292 STRIPNAME(striptype));
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6293 return NULL;
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6294 }
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6295 }
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6296
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6297 return do_strip(self, striptype);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6298 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6299
23225 e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6300
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	6301 PyDoc_STRVAR(strip__doc__,
28575 215f1f4851a4 Attempt to make all the various string strip methods the same. Neal Norwitz <nnorwitz@gmail.com>* parents: 28559 diff changeset	6302 "S.strip([chars]) -> unicode\n\
23225 e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6303 \n\
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6304 Return a copy of the string S with leading and trailing\n\
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6305 whitespace removed.\n\
28575 215f1f4851a4 Attempt to make all the various string strip methods the same. Neal Norwitz <nnorwitz@gmail.com>* parents: 28559 diff changeset	6306 If chars is given and not None, remove characters in chars instead.\n\
215f1f4851a4 Attempt to make all the various string strip methods the same. Neal Norwitz <nnorwitz@gmail.com>* parents: 28559 diff changeset	6307 If chars is a str, it will be converted to unicode before stripping");
23225 e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6308
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6309 static PyObject *
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6310 unicode_strip(PyUnicodeObject self, PyObject args)
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6311 {
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6312 if (PyTuple_GET_SIZE(args) == 0)
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6313 return do_strip(self, BOTHSTRIP); /* Common case */
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6314 else
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6315 return do_argstrip(self, BOTHSTRIP, args);
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6316 }
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6317
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6318
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	6319 PyDoc_STRVAR(lstrip__doc__,
28575 215f1f4851a4 Attempt to make all the various string strip methods the same. Neal Norwitz <nnorwitz@gmail.com>* parents: 28559 diff changeset	6320 "S.lstrip([chars]) -> unicode\n\
23225 e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6321 \n\
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6322 Return a copy of the string S with leading whitespace removed.\n\
28575 215f1f4851a4 Attempt to make all the various string strip methods the same. Neal Norwitz <nnorwitz@gmail.com>* parents: 28559 diff changeset	6323 If chars is given and not None, remove characters in chars instead.\n\
215f1f4851a4 Attempt to make all the various string strip methods the same. Neal Norwitz <nnorwitz@gmail.com>* parents: 28559 diff changeset	6324 If chars is a str, it will be converted to unicode before stripping");
23225 e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6325
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6326 static PyObject *
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6327 unicode_lstrip(PyUnicodeObject self, PyObject args)
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6328 {
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6329 if (PyTuple_GET_SIZE(args) == 0)
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6330 return do_strip(self, LEFTSTRIP); /* Common case */
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6331 else
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6332 return do_argstrip(self, LEFTSTRIP, args);
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6333 }
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6334
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6335
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	6336 PyDoc_STRVAR(rstrip__doc__,
28575 215f1f4851a4 Attempt to make all the various string strip methods the same. Neal Norwitz <nnorwitz@gmail.com>* parents: 28559 diff changeset	6337 "S.rstrip([chars]) -> unicode\n\
23225 e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6338 \n\
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6339 Return a copy of the string S with trailing whitespace removed.\n\
28575 215f1f4851a4 Attempt to make all the various string strip methods the same. Neal Norwitz <nnorwitz@gmail.com>* parents: 28559 diff changeset	6340 If chars is given and not None, remove characters in chars instead.\n\
215f1f4851a4 Attempt to make all the various string strip methods the same. Neal Norwitz <nnorwitz@gmail.com>* parents: 28559 diff changeset	6341 If chars is a str, it will be converted to unicode before stripping");
23225 e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6342
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6343 static PyObject *
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6344 unicode_rstrip(PyUnicodeObject self, PyObject args)
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6345 {
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6346 if (PyTuple_GET_SIZE(args) == 0)
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6347 return do_strip(self, RIGHTSTRIP); /* Common case */
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6348 else
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6349 return do_argstrip(self, RIGHTSTRIP, args);
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6350 }
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6351
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	6352
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6353 static PyObject*
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	6354 unicode_repeat(PyUnicodeObject *str, Py_ssize_t len)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6355 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6356 PyUnicodeObject *u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6357 Py_UNICODE *p;
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	6358 Py_ssize_t nchars;
14492 7728b0ea9344 Fix for bug 113934. stringn and unicoden did no overflow checking at Tim Peters <tim.peters@gmail.com> parents: 14388 diff changeset	6359 size_t nbytes;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6360
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6361 if (len < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6362 len = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6363
19885 72e6eb653312 More on bug 460020: disable many optimizations of unicode subclasses. Tim Peters <tim.peters@gmail.com> parents: 19849 diff changeset	6364 if (len == 1 && PyUnicode_CheckExact(str)) {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6365 /* no repeat, return original string */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6366 Py_INCREF(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6367 return (PyObject*) str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6368 }
14492 7728b0ea9344 Fix for bug 113934. stringn and unicoden did no overflow checking at Tim Peters <tim.peters@gmail.com> parents: 14388 diff changeset	6369
7728b0ea9344 Fix for bug 113934. stringn and unicoden did no overflow checking at Tim Peters <tim.peters@gmail.com> parents: 14388 diff changeset	6370 /* ensure # of chars needed doesn't overflow int and # of bytes
7728b0ea9344 Fix for bug 113934. stringn and unicoden did no overflow checking at Tim Peters <tim.peters@gmail.com> parents: 14388 diff changeset	6371 * needed doesn't overflow size_t
7728b0ea9344 Fix for bug 113934. stringn and unicoden did no overflow checking at Tim Peters <tim.peters@gmail.com> parents: 14388 diff changeset	6372 */
7728b0ea9344 Fix for bug 113934. stringn and unicoden did no overflow checking at Tim Peters <tim.peters@gmail.com> parents: 14388 diff changeset	6373 nchars = len * str->length;
7728b0ea9344 Fix for bug 113934. stringn and unicoden did no overflow checking at Tim Peters <tim.peters@gmail.com> parents: 14388 diff changeset	6374 if (len && nchars / len != str->length) {
7728b0ea9344 Fix for bug 113934. stringn and unicoden did no overflow checking at Tim Peters <tim.peters@gmail.com> parents: 14388 diff changeset	6375 PyErr_SetString(PyExc_OverflowError,
7728b0ea9344 Fix for bug 113934. stringn and unicoden did no overflow checking at Tim Peters <tim.peters@gmail.com> parents: 14388 diff changeset	6376 "repeated string is too long");
7728b0ea9344 Fix for bug 113934. stringn and unicoden did no overflow checking at Tim Peters <tim.peters@gmail.com> parents: 14388 diff changeset	6377 return NULL;
7728b0ea9344 Fix for bug 113934. stringn and unicoden did no overflow checking at Tim Peters <tim.peters@gmail.com> parents: 14388 diff changeset	6378 }
7728b0ea9344 Fix for bug 113934. stringn and unicoden did no overflow checking at Tim Peters <tim.peters@gmail.com> parents: 14388 diff changeset	6379 nbytes = (nchars + 1) * sizeof(Py_UNICODE);
7728b0ea9344 Fix for bug 113934. stringn and unicoden did no overflow checking at Tim Peters <tim.peters@gmail.com> parents: 14388 diff changeset	6380 if (nbytes / sizeof(Py_UNICODE) != (size_t)(nchars + 1)) {
7728b0ea9344 Fix for bug 113934. stringn and unicoden did no overflow checking at Tim Peters <tim.peters@gmail.com> parents: 14388 diff changeset	6381 PyErr_SetString(PyExc_OverflowError,
7728b0ea9344 Fix for bug 113934. stringn and unicoden did no overflow checking at Tim Peters <tim.peters@gmail.com> parents: 14388 diff changeset	6382 "repeated string is too long");
7728b0ea9344 Fix for bug 113934. stringn and unicoden did no overflow checking at Tim Peters <tim.peters@gmail.com> parents: 14388 diff changeset	6383 return NULL;
7728b0ea9344 Fix for bug 113934. stringn and unicoden did no overflow checking at Tim Peters <tim.peters@gmail.com> parents: 14388 diff changeset	6384 }
7728b0ea9344 Fix for bug 113934. stringn and unicoden did no overflow checking at Tim Peters <tim.peters@gmail.com> parents: 14388 diff changeset	6385 u = _PyUnicode_New(nchars);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6386 if (!u)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6387 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6388
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6389 p = u->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6390
38238 feb00df91922 needforspeed: speed up unicode repeat, unicode string copy Fredrik Lundh <fredrik@pythonware.com> parents: 38236 diff changeset	6391 if (str->length == 1 && len > 0) {
feb00df91922 needforspeed: speed up unicode repeat, unicode string copy Fredrik Lundh <fredrik@pythonware.com> parents: 38236 diff changeset	6392 Py_UNICODE_FILL(p, str->str[0], len);
38239 604af867c65f needforspeed: use memcpy for "long" strings; use a better algorithm Fredrik Lundh <fredrik@pythonware.com> parents: 38238 diff changeset	6393 } else {
38241 5eaf8b9587d8 unicode_repeat(): Change type of local to Py_ssize_t, Tim Peters <tim.peters@gmail.com> parents: 38240 diff changeset	6394 Py_ssize_t done = 0; /* number of characters copied this far */
38239 604af867c65f needforspeed: use memcpy for "long" strings; use a better algorithm Fredrik Lundh <fredrik@pythonware.com> parents: 38238 diff changeset	6395 if (done < nchars) {
38238 feb00df91922 needforspeed: speed up unicode repeat, unicode string copy Fredrik Lundh <fredrik@pythonware.com> parents: 38236 diff changeset	6396 Py_UNICODE_COPY(p, str->str, str->length);
38239 604af867c65f needforspeed: use memcpy for "long" strings; use a better algorithm Fredrik Lundh <fredrik@pythonware.com> parents: 38238 diff changeset	6397 done = str->length;
604af867c65f needforspeed: use memcpy for "long" strings; use a better algorithm Fredrik Lundh <fredrik@pythonware.com> parents: 38238 diff changeset	6398 }
604af867c65f needforspeed: use memcpy for "long" strings; use a better algorithm Fredrik Lundh <fredrik@pythonware.com> parents: 38238 diff changeset	6399 while (done < nchars) {
604af867c65f needforspeed: use memcpy for "long" strings; use a better algorithm Fredrik Lundh <fredrik@pythonware.com> parents: 38238 diff changeset	6400 int n = (done <= nchars-done) ? done : nchars-done;
604af867c65f needforspeed: use memcpy for "long" strings; use a better algorithm Fredrik Lundh <fredrik@pythonware.com> parents: 38238 diff changeset	6401 Py_UNICODE_COPY(p+done, p, n);
604af867c65f needforspeed: use memcpy for "long" strings; use a better algorithm Fredrik Lundh <fredrik@pythonware.com> parents: 38238 diff changeset	6402 done += n;
604af867c65f needforspeed: use memcpy for "long" strings; use a better algorithm Fredrik Lundh <fredrik@pythonware.com> parents: 38238 diff changeset	6403 }
604af867c65f needforspeed: use memcpy for "long" strings; use a better algorithm Fredrik Lundh <fredrik@pythonware.com> parents: 38238 diff changeset	6404 }
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6405
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6406 return (PyObject*) u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6407 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6408
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6409 PyObject PyUnicode_Replace(PyObject obj,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6410 PyObject *subobj,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6411 PyObject *replobj,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	6412 Py_ssize_t maxcount)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6413 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6414 PyObject *self;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6415 PyObject *str1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6416 PyObject *str2;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6417 PyObject *result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6418
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6419 self = PyUnicode_FromObject(obj);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6420 if (self == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6421 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6422 str1 = PyUnicode_FromObject(subobj);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6423 if (str1 == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6424 Py_DECREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6425 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6426 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6427 str2 = PyUnicode_FromObject(replobj);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6428 if (str2 == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6429 Py_DECREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6430 Py_DECREF(str1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6431 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6432 }
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	6433 result = replace((PyUnicodeObject *)self,
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	6434 (PyUnicodeObject *)str1,
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	6435 (PyUnicodeObject *)str2,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6436 maxcount);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6437 Py_DECREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6438 Py_DECREF(str1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6439 Py_DECREF(str2);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6440 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6441 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6442
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	6443 PyDoc_STRVAR(replace__doc__,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6444 "S.replace (old, new[, maxsplit]) -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6445 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6446 Return a copy of S with all occurrences of substring\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6447 old replaced by new. If the optional argument maxsplit is\n\
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	6448 given, only the first maxsplit occurrences are replaced.");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6449
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6450 static PyObject*
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6451 unicode_replace(PyUnicodeObject self, PyObject args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6452 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6453 PyUnicodeObject *str1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6454 PyUnicodeObject *str2;
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	6455 Py_ssize_t maxcount = -1;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6456 PyObject *result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6457
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	6458 if (!PyArg_ParseTuple(args, "OO\|n:replace", &str1, &str2, &maxcount))
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6459 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6460 str1 = (PyUnicodeObject )PyUnicode_FromObject((PyObject )str1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6461 if (str1 == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6462 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6463 str2 = (PyUnicodeObject )PyUnicode_FromObject((PyObject )str2);
27774 bfc524379307 Fix two refcounting bugs Walter Dörwald <walter@livinglogic.de> parents: 27555 diff changeset	6464 if (str2 == NULL) {
bfc524379307 Fix two refcounting bugs Walter Dörwald <walter@livinglogic.de> parents: 27555 diff changeset	6465 Py_DECREF(str1);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6466 return NULL;
27774 bfc524379307 Fix two refcounting bugs Walter Dörwald <walter@livinglogic.de> parents: 27555 diff changeset	6467 }
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6468
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6469 result = replace(self, str1, str2, maxcount);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6470
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6471 Py_DECREF(str1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6472 Py_DECREF(str2);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6473 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6474 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6475
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6476 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6477 PyObject unicode_repr(PyObject unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6478 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6479 return unicodeescape_string(PyUnicode_AS_UNICODE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6480 PyUnicode_GET_SIZE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6481 1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6482 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6483
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	6484 PyDoc_STRVAR(rfind__doc__,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6485 "S.rfind(sub [,start [,end]]) -> int\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6486 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6487 Return the highest index in S where substring sub is found,\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6488 such that sub is contained within s[start,end]. Optional\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6489 arguments start and end are interpreted as in slice notation.\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6490 \n\
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	6491 Return -1 on failure.");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6492
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6493 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6494 unicode_rfind(PyUnicodeObject self, PyObject args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6495 {
38424 3d81faf47af7 needforspeed: more stringlib refactoring Fredrik Lundh <fredrik@pythonware.com> parents: 38422 diff changeset	6496 PyObject *substring;
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	6497 Py_ssize_t start = 0;
37662 0678ce9fb6b2 Change more ints to Py_ssize_t. Martin v. Löwis <martin@v.loewis.de> parents: 37661 diff changeset	6498 Py_ssize_t end = PY_SSIZE_T_MAX;
38424 3d81faf47af7 needforspeed: more stringlib refactoring Fredrik Lundh <fredrik@pythonware.com> parents: 38422 diff changeset	6499 Py_ssize_t result;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6500
12629 3efeaa7521e4 Trent Mick: Guido van Rossum <guido@python.org> parents: 12582 diff changeset	6501 if (!PyArg_ParseTuple(args, "O\|O&O&:rfind", &substring,
3efeaa7521e4 Trent Mick: Guido van Rossum <guido@python.org> parents: 12582 diff changeset	6502 _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end))
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6503 return NULL;
38424 3d81faf47af7 needforspeed: more stringlib refactoring Fredrik Lundh <fredrik@pythonware.com> parents: 38422 diff changeset	6504 substring = PyUnicode_FromObject(substring);
3d81faf47af7 needforspeed: more stringlib refactoring Fredrik Lundh <fredrik@pythonware.com> parents: 38422 diff changeset	6505 if (!substring)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6506 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6507
38441 72327f8f4a79 needforspeed: stringlib refactoring: changed find_obj to find_slice, Fredrik Lundh <fredrik@pythonware.com> parents: 38440 diff changeset	6508 result = stringlib_rfind_slice(
72327f8f4a79 needforspeed: stringlib refactoring: changed find_obj to find_slice, Fredrik Lundh <fredrik@pythonware.com> parents: 38440 diff changeset	6509 PyUnicode_AS_UNICODE(self), PyUnicode_GET_SIZE(self),
72327f8f4a79 needforspeed: stringlib refactoring: changed find_obj to find_slice, Fredrik Lundh <fredrik@pythonware.com> parents: 38440 diff changeset	6510 PyUnicode_AS_UNICODE(substring), PyUnicode_GET_SIZE(substring),
72327f8f4a79 needforspeed: stringlib refactoring: changed find_obj to find_slice, Fredrik Lundh <fredrik@pythonware.com> parents: 38440 diff changeset	6511 start, end
72327f8f4a79 needforspeed: stringlib refactoring: changed find_obj to find_slice, Fredrik Lundh <fredrik@pythonware.com> parents: 38440 diff changeset	6512 );
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6513
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6514 Py_DECREF(substring);
38424 3d81faf47af7 needforspeed: more stringlib refactoring Fredrik Lundh <fredrik@pythonware.com> parents: 38422 diff changeset	6515
3d81faf47af7 needforspeed: more stringlib refactoring Fredrik Lundh <fredrik@pythonware.com> parents: 38422 diff changeset	6516 return PyInt_FromSsize_t(result);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6517 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6518
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	6519 PyDoc_STRVAR(rindex__doc__,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6520 "S.rindex(sub [,start [,end]]) -> int\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6521 \n\
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	6522 Like S.rfind() but raise ValueError when the substring is not found.");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6523
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6524 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6525 unicode_rindex(PyUnicodeObject self, PyObject args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6526 {
38424 3d81faf47af7 needforspeed: more stringlib refactoring Fredrik Lundh <fredrik@pythonware.com> parents: 38422 diff changeset	6527 PyObject *substring;
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	6528 Py_ssize_t start = 0;
37662 0678ce9fb6b2 Change more ints to Py_ssize_t. Martin v. Löwis <martin@v.loewis.de> parents: 37661 diff changeset	6529 Py_ssize_t end = PY_SSIZE_T_MAX;
38424 3d81faf47af7 needforspeed: more stringlib refactoring Fredrik Lundh <fredrik@pythonware.com> parents: 38422 diff changeset	6530 Py_ssize_t result;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6531
12629 3efeaa7521e4 Trent Mick: Guido van Rossum <guido@python.org> parents: 12582 diff changeset	6532 if (!PyArg_ParseTuple(args, "O\|O&O&:rindex", &substring,
3efeaa7521e4 Trent Mick: Guido van Rossum <guido@python.org> parents: 12582 diff changeset	6533 _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end))
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6534 return NULL;
38424 3d81faf47af7 needforspeed: more stringlib refactoring Fredrik Lundh <fredrik@pythonware.com> parents: 38422 diff changeset	6535 substring = PyUnicode_FromObject(substring);
3d81faf47af7 needforspeed: more stringlib refactoring Fredrik Lundh <fredrik@pythonware.com> parents: 38422 diff changeset	6536 if (!substring)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6537 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6538
38441 72327f8f4a79 needforspeed: stringlib refactoring: changed find_obj to find_slice, Fredrik Lundh <fredrik@pythonware.com> parents: 38440 diff changeset	6539 result = stringlib_rfind_slice(
72327f8f4a79 needforspeed: stringlib refactoring: changed find_obj to find_slice, Fredrik Lundh <fredrik@pythonware.com> parents: 38440 diff changeset	6540 PyUnicode_AS_UNICODE(self), PyUnicode_GET_SIZE(self),
72327f8f4a79 needforspeed: stringlib refactoring: changed find_obj to find_slice, Fredrik Lundh <fredrik@pythonware.com> parents: 38440 diff changeset	6541 PyUnicode_AS_UNICODE(substring), PyUnicode_GET_SIZE(substring),
72327f8f4a79 needforspeed: stringlib refactoring: changed find_obj to find_slice, Fredrik Lundh <fredrik@pythonware.com> parents: 38440 diff changeset	6542 start, end
72327f8f4a79 needforspeed: stringlib refactoring: changed find_obj to find_slice, Fredrik Lundh <fredrik@pythonware.com> parents: 38440 diff changeset	6543 );
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6544
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6545 Py_DECREF(substring);
38424 3d81faf47af7 needforspeed: more stringlib refactoring Fredrik Lundh <fredrik@pythonware.com> parents: 38422 diff changeset	6546
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6547 if (result < 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6548 PyErr_SetString(PyExc_ValueError, "substring not found");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6549 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6550 }
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	6551 return PyInt_FromSsize_t(result);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6552 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6553
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	6554 PyDoc_STRVAR(rjust__doc__,
31021 21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods. Raymond Hettinger <python@rcn.com> parents: 30765 diff changeset	6555 "S.rjust(width[, fillchar]) -> unicode\n\
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6556 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6557 Return S right justified in a Unicode string of length width. Padding is\n\
31021 21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods. Raymond Hettinger <python@rcn.com> parents: 30765 diff changeset	6558 done using the specified fill character (default is a space).");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6559
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6560 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6561 unicode_rjust(PyUnicodeObject self, PyObject args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6562 {
37662 0678ce9fb6b2 Change more ints to Py_ssize_t. Martin v. Löwis <martin@v.loewis.de> parents: 37661 diff changeset	6563 Py_ssize_t width;
31021 21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods. Raymond Hettinger <python@rcn.com> parents: 30765 diff changeset	6564 Py_UNICODE fillchar = ' ';
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods. Raymond Hettinger <python@rcn.com> parents: 30765 diff changeset	6565
37662 0678ce9fb6b2 Change more ints to Py_ssize_t. Martin v. Löwis <martin@v.loewis.de> parents: 37661 diff changeset	6566 if (!PyArg_ParseTuple(args, "n\|O&:rjust", &width, convert_uc, &fillchar))
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6567 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6568
19885 72e6eb653312 More on bug 460020: disable many optimizations of unicode subclasses. Tim Peters <tim.peters@gmail.com> parents: 19849 diff changeset	6569 if (self->length >= width && PyUnicode_CheckExact(self)) {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6570 Py_INCREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6571 return (PyObject*) self;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6572 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6573
31021 21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods. Raymond Hettinger <python@rcn.com> parents: 30765 diff changeset	6574 return (PyObject*) pad(self, width - self->length, 0, fillchar);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6575 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6576
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6577 static PyObject*
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	6578 unicode_slice(PyUnicodeObject *self, Py_ssize_t start, Py_ssize_t end)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6579 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6580 /* standard clamping */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6581 if (start < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6582 start = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6583 if (end < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6584 end = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6585 if (end > self->length)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6586 end = self->length;
19885 72e6eb653312 More on bug 460020: disable many optimizations of unicode subclasses. Tim Peters <tim.peters@gmail.com> parents: 19849 diff changeset	6587 if (start == 0 && end == self->length && PyUnicode_CheckExact(self)) {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6588 /* full slice, return original string */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6589 Py_INCREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6590 return (PyObject*) self;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6591 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6592 if (start > end)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6593 start = end;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6594 /* copy slice */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6595 return (PyObject*) PyUnicode_FromUnicode(self->str + start,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6596 end - start);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6597 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6598
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6599 PyObject PyUnicode_Split(PyObject s,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6600 PyObject *sep,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	6601 Py_ssize_t maxsplit)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6602 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6603 PyObject *result;
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	6604
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6605 s = PyUnicode_FromObject(s);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6606 if (s == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6607 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6608 if (sep != NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6609 sep = PyUnicode_FromObject(sep);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6610 if (sep == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6611 Py_DECREF(s);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6612 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6613 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6614 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6615
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6616 result = split((PyUnicodeObject )s, (PyUnicodeObject )sep, maxsplit);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6617
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6618 Py_DECREF(s);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6619 Py_XDECREF(sep);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6620 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6621 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6622
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	6623 PyDoc_STRVAR(split__doc__,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6624 "S.split([sep [,maxsplit]]) -> list of strings\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6625 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6626 Return a list of the words in S, using sep as the\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6627 delimiter string. If maxsplit is given, at most maxsplit\n\
33495 2d6e1854082e Add a missing line continuation character. Thomas Heller <theller@ctypes.org> parents: 33487 diff changeset	6628 splits are done. If sep is not specified or is None,\n\
33486 53c1357c9c63 Enhance the docstrings for unicode.split() and string.split() Walter Dörwald <walter@livinglogic.de> parents: 33428 diff changeset	6629 any whitespace string is a separator.");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6630
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6631 static PyObject*
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6632 unicode_split(PyUnicodeObject self, PyObject args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6633 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6634 PyObject *substring = Py_None;
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	6635 Py_ssize_t maxcount = -1;
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	6636
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	6637 if (!PyArg_ParseTuple(args, "\|On:split", &substring, &maxcount))
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6638 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6639
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6640 if (substring == Py_None)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6641 return split(self, NULL, maxcount);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6642 else if (PyUnicode_Check(substring))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6643 return split(self, (PyUnicodeObject *)substring, maxcount);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6644 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6645 return PyUnicode_Split((PyObject *)self, substring, maxcount);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6646 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6647
38335 f9332ad4287a needforspeed: partition implementation, part two. Fredrik Lundh <fredrik@pythonware.com> parents: 38304 diff changeset	6648 PyObject *
f9332ad4287a needforspeed: partition implementation, part two. Fredrik Lundh <fredrik@pythonware.com> parents: 38304 diff changeset	6649 PyUnicode_Partition(PyObject str_in, PyObject sep_in)
f9332ad4287a needforspeed: partition implementation, part two. Fredrik Lundh <fredrik@pythonware.com> parents: 38304 diff changeset	6650 {
f9332ad4287a needforspeed: partition implementation, part two. Fredrik Lundh <fredrik@pythonware.com> parents: 38304 diff changeset	6651 PyObject* str_obj;
f9332ad4287a needforspeed: partition implementation, part two. Fredrik Lundh <fredrik@pythonware.com> parents: 38304 diff changeset	6652 PyObject* sep_obj;
f9332ad4287a needforspeed: partition implementation, part two. Fredrik Lundh <fredrik@pythonware.com> parents: 38304 diff changeset	6653 PyObject* out;
38383 05744917b7c3 needforspeed: stringlib refactoring (in progress) Fredrik Lundh <fredrik@pythonware.com> parents: 38381 diff changeset	6654
38335 f9332ad4287a needforspeed: partition implementation, part two. Fredrik Lundh <fredrik@pythonware.com> parents: 38304 diff changeset	6655 str_obj = PyUnicode_FromObject(str_in);
f9332ad4287a needforspeed: partition implementation, part two. Fredrik Lundh <fredrik@pythonware.com> parents: 38304 diff changeset	6656 if (!str_obj)
f9332ad4287a needforspeed: partition implementation, part two. Fredrik Lundh <fredrik@pythonware.com> parents: 38304 diff changeset	6657 return NULL;
f9332ad4287a needforspeed: partition implementation, part two. Fredrik Lundh <fredrik@pythonware.com> parents: 38304 diff changeset	6658 sep_obj = PyUnicode_FromObject(sep_in);
38383 05744917b7c3 needforspeed: stringlib refactoring (in progress) Fredrik Lundh <fredrik@pythonware.com> parents: 38381 diff changeset	6659 if (!sep_obj) {
05744917b7c3 needforspeed: stringlib refactoring (in progress) Fredrik Lundh <fredrik@pythonware.com> parents: 38381 diff changeset	6660 Py_DECREF(str_obj);
05744917b7c3 needforspeed: stringlib refactoring (in progress) Fredrik Lundh <fredrik@pythonware.com> parents: 38381 diff changeset	6661 return NULL;
05744917b7c3 needforspeed: stringlib refactoring (in progress) Fredrik Lundh <fredrik@pythonware.com> parents: 38381 diff changeset	6662 }
05744917b7c3 needforspeed: stringlib refactoring (in progress) Fredrik Lundh <fredrik@pythonware.com> parents: 38381 diff changeset	6663
38400 33c328896f52 needforspeed: stringlib refactoring, continued. added count and Fredrik Lundh <fredrik@pythonware.com> parents: 38391 diff changeset	6664 out = stringlib_partition(
38383 05744917b7c3 needforspeed: stringlib refactoring (in progress) Fredrik Lundh <fredrik@pythonware.com> parents: 38381 diff changeset	6665 str_obj, PyUnicode_AS_UNICODE(str_obj), PyUnicode_GET_SIZE(str_obj),
05744917b7c3 needforspeed: stringlib refactoring (in progress) Fredrik Lundh <fredrik@pythonware.com> parents: 38381 diff changeset	6666 sep_obj, PyUnicode_AS_UNICODE(sep_obj), PyUnicode_GET_SIZE(sep_obj)
05744917b7c3 needforspeed: stringlib refactoring (in progress) Fredrik Lundh <fredrik@pythonware.com> parents: 38381 diff changeset	6667 );
05744917b7c3 needforspeed: stringlib refactoring (in progress) Fredrik Lundh <fredrik@pythonware.com> parents: 38381 diff changeset	6668
05744917b7c3 needforspeed: stringlib refactoring (in progress) Fredrik Lundh <fredrik@pythonware.com> parents: 38381 diff changeset	6669 Py_DECREF(sep_obj);
05744917b7c3 needforspeed: stringlib refactoring (in progress) Fredrik Lundh <fredrik@pythonware.com> parents: 38381 diff changeset	6670 Py_DECREF(str_obj);
38335 f9332ad4287a needforspeed: partition implementation, part two. Fredrik Lundh <fredrik@pythonware.com> parents: 38304 diff changeset	6671
f9332ad4287a needforspeed: partition implementation, part two. Fredrik Lundh <fredrik@pythonware.com> parents: 38304 diff changeset	6672 return out;
f9332ad4287a needforspeed: partition implementation, part two. Fredrik Lundh <fredrik@pythonware.com> parents: 38304 diff changeset	6673 }
f9332ad4287a needforspeed: partition implementation, part two. Fredrik Lundh <fredrik@pythonware.com> parents: 38304 diff changeset	6674
38390 76807902a2f5 needforspeed: added rpartition implementation Fredrik Lundh <fredrik@pythonware.com> parents: 38383 diff changeset	6675
76807902a2f5 needforspeed: added rpartition implementation Fredrik Lundh <fredrik@pythonware.com> parents: 38383 diff changeset	6676 PyObject *
76807902a2f5 needforspeed: added rpartition implementation Fredrik Lundh <fredrik@pythonware.com> parents: 38383 diff changeset	6677 PyUnicode_RPartition(PyObject str_in, PyObject sep_in)
76807902a2f5 needforspeed: added rpartition implementation Fredrik Lundh <fredrik@pythonware.com> parents: 38383 diff changeset	6678 {
76807902a2f5 needforspeed: added rpartition implementation Fredrik Lundh <fredrik@pythonware.com> parents: 38383 diff changeset	6679 PyObject* str_obj;
76807902a2f5 needforspeed: added rpartition implementation Fredrik Lundh <fredrik@pythonware.com> parents: 38383 diff changeset	6680 PyObject* sep_obj;
76807902a2f5 needforspeed: added rpartition implementation Fredrik Lundh <fredrik@pythonware.com> parents: 38383 diff changeset	6681 PyObject* out;
76807902a2f5 needforspeed: added rpartition implementation Fredrik Lundh <fredrik@pythonware.com> parents: 38383 diff changeset	6682
76807902a2f5 needforspeed: added rpartition implementation Fredrik Lundh <fredrik@pythonware.com> parents: 38383 diff changeset	6683 str_obj = PyUnicode_FromObject(str_in);
76807902a2f5 needforspeed: added rpartition implementation Fredrik Lundh <fredrik@pythonware.com> parents: 38383 diff changeset	6684 if (!str_obj)
76807902a2f5 needforspeed: added rpartition implementation Fredrik Lundh <fredrik@pythonware.com> parents: 38383 diff changeset	6685 return NULL;
76807902a2f5 needforspeed: added rpartition implementation Fredrik Lundh <fredrik@pythonware.com> parents: 38383 diff changeset	6686 sep_obj = PyUnicode_FromObject(sep_in);
76807902a2f5 needforspeed: added rpartition implementation Fredrik Lundh <fredrik@pythonware.com> parents: 38383 diff changeset	6687 if (!sep_obj) {
76807902a2f5 needforspeed: added rpartition implementation Fredrik Lundh <fredrik@pythonware.com> parents: 38383 diff changeset	6688 Py_DECREF(str_obj);
76807902a2f5 needforspeed: added rpartition implementation Fredrik Lundh <fredrik@pythonware.com> parents: 38383 diff changeset	6689 return NULL;
76807902a2f5 needforspeed: added rpartition implementation Fredrik Lundh <fredrik@pythonware.com> parents: 38383 diff changeset	6690 }
76807902a2f5 needforspeed: added rpartition implementation Fredrik Lundh <fredrik@pythonware.com> parents: 38383 diff changeset	6691
38400 33c328896f52 needforspeed: stringlib refactoring, continued. added count and Fredrik Lundh <fredrik@pythonware.com> parents: 38391 diff changeset	6692 out = stringlib_rpartition(
38390 76807902a2f5 needforspeed: added rpartition implementation Fredrik Lundh <fredrik@pythonware.com> parents: 38383 diff changeset	6693 str_obj, PyUnicode_AS_UNICODE(str_obj), PyUnicode_GET_SIZE(str_obj),
76807902a2f5 needforspeed: added rpartition implementation Fredrik Lundh <fredrik@pythonware.com> parents: 38383 diff changeset	6694 sep_obj, PyUnicode_AS_UNICODE(sep_obj), PyUnicode_GET_SIZE(sep_obj)
76807902a2f5 needforspeed: added rpartition implementation Fredrik Lundh <fredrik@pythonware.com> parents: 38383 diff changeset	6695 );
76807902a2f5 needforspeed: added rpartition implementation Fredrik Lundh <fredrik@pythonware.com> parents: 38383 diff changeset	6696
76807902a2f5 needforspeed: added rpartition implementation Fredrik Lundh <fredrik@pythonware.com> parents: 38383 diff changeset	6697 Py_DECREF(sep_obj);
76807902a2f5 needforspeed: added rpartition implementation Fredrik Lundh <fredrik@pythonware.com> parents: 38383 diff changeset	6698 Py_DECREF(str_obj);
76807902a2f5 needforspeed: added rpartition implementation Fredrik Lundh <fredrik@pythonware.com> parents: 38383 diff changeset	6699
76807902a2f5 needforspeed: added rpartition implementation Fredrik Lundh <fredrik@pythonware.com> parents: 38383 diff changeset	6700 return out;
76807902a2f5 needforspeed: added rpartition implementation Fredrik Lundh <fredrik@pythonware.com> parents: 38383 diff changeset	6701 }
76807902a2f5 needforspeed: added rpartition implementation Fredrik Lundh <fredrik@pythonware.com> parents: 38383 diff changeset	6702
38335 f9332ad4287a needforspeed: partition implementation, part two. Fredrik Lundh <fredrik@pythonware.com> parents: 38304 diff changeset	6703 PyDoc_STRVAR(partition__doc__,
f9332ad4287a needforspeed: partition implementation, part two. Fredrik Lundh <fredrik@pythonware.com> parents: 38304 diff changeset	6704 "S.partition(sep) -> (head, sep, tail)\n\
f9332ad4287a needforspeed: partition implementation, part two. Fredrik Lundh <fredrik@pythonware.com> parents: 38304 diff changeset	6705 \n\
f9332ad4287a needforspeed: partition implementation, part two. Fredrik Lundh <fredrik@pythonware.com> parents: 38304 diff changeset	6706 Searches for the separator sep in S, and returns the part before it,\n\
f9332ad4287a needforspeed: partition implementation, part two. Fredrik Lundh <fredrik@pythonware.com> parents: 38304 diff changeset	6707 the separator itself, and the part after it. If the separator is not\n\
f9332ad4287a needforspeed: partition implementation, part two. Fredrik Lundh <fredrik@pythonware.com> parents: 38304 diff changeset	6708 found, returns S and two empty strings.");
f9332ad4287a needforspeed: partition implementation, part two. Fredrik Lundh <fredrik@pythonware.com> parents: 38304 diff changeset	6709
f9332ad4287a needforspeed: partition implementation, part two. Fredrik Lundh <fredrik@pythonware.com> parents: 38304 diff changeset	6710 static PyObject*
38338 958f91684209 needforspeed: use METH_O for argument handling, which made partition some Fredrik Lundh <fredrik@pythonware.com> parents: 38335 diff changeset	6711 unicode_partition(PyUnicodeObject self, PyObject separator)
958f91684209 needforspeed: use METH_O for argument handling, which made partition some Fredrik Lundh <fredrik@pythonware.com> parents: 38335 diff changeset	6712 {
38335 f9332ad4287a needforspeed: partition implementation, part two. Fredrik Lundh <fredrik@pythonware.com> parents: 38304 diff changeset	6713 return PyUnicode_Partition((PyObject *)self, separator);
f9332ad4287a needforspeed: partition implementation, part two. Fredrik Lundh <fredrik@pythonware.com> parents: 38304 diff changeset	6714 }
f9332ad4287a needforspeed: partition implementation, part two. Fredrik Lundh <fredrik@pythonware.com> parents: 38304 diff changeset	6715
38390 76807902a2f5 needforspeed: added rpartition implementation Fredrik Lundh <fredrik@pythonware.com> parents: 38383 diff changeset	6716 PyDoc_STRVAR(rpartition__doc__,
39734 cfb77fff0240 Fix endcase for str.rpartition() Raymond Hettinger <python@rcn.com> parents: 39621 diff changeset	6717 "S.rpartition(sep) -> (tail, sep, head)\n\
38390 76807902a2f5 needforspeed: added rpartition implementation Fredrik Lundh <fredrik@pythonware.com> parents: 38383 diff changeset	6718 \n\
76807902a2f5 needforspeed: added rpartition implementation Fredrik Lundh <fredrik@pythonware.com> parents: 38383 diff changeset	6719 Searches for the separator sep in S, starting at the end of S, and returns\n\
76807902a2f5 needforspeed: added rpartition implementation Fredrik Lundh <fredrik@pythonware.com> parents: 38383 diff changeset	6720 the part before it, the separator itself, and the part after it. If the\n\
39734 cfb77fff0240 Fix endcase for str.rpartition() Raymond Hettinger <python@rcn.com> parents: 39621 diff changeset	6721 separator is not found, returns two empty strings and S.");
38390 76807902a2f5 needforspeed: added rpartition implementation Fredrik Lundh <fredrik@pythonware.com> parents: 38383 diff changeset	6722
76807902a2f5 needforspeed: added rpartition implementation Fredrik Lundh <fredrik@pythonware.com> parents: 38383 diff changeset	6723 static PyObject*
76807902a2f5 needforspeed: added rpartition implementation Fredrik Lundh <fredrik@pythonware.com> parents: 38383 diff changeset	6724 unicode_rpartition(PyUnicodeObject self, PyObject separator)
76807902a2f5 needforspeed: added rpartition implementation Fredrik Lundh <fredrik@pythonware.com> parents: 38383 diff changeset	6725 {
76807902a2f5 needforspeed: added rpartition implementation Fredrik Lundh <fredrik@pythonware.com> parents: 38383 diff changeset	6726 return PyUnicode_RPartition((PyObject *)self, separator);
76807902a2f5 needforspeed: added rpartition implementation Fredrik Lundh <fredrik@pythonware.com> parents: 38383 diff changeset	6727 }
76807902a2f5 needforspeed: added rpartition implementation Fredrik Lundh <fredrik@pythonware.com> parents: 38383 diff changeset	6728
31199 504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	6729 PyObject PyUnicode_RSplit(PyObject s,
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	6730 PyObject *sep,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	6731 Py_ssize_t maxsplit)
31199 504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	6732 {
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	6733 PyObject *result;
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	6734
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	6735 s = PyUnicode_FromObject(s);
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	6736 if (s == NULL)
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	6737 return NULL;
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	6738 if (sep != NULL) {
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	6739 sep = PyUnicode_FromObject(sep);
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	6740 if (sep == NULL) {
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	6741 Py_DECREF(s);
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	6742 return NULL;
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	6743 }
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	6744 }
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	6745
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	6746 result = rsplit((PyUnicodeObject )s, (PyUnicodeObject )sep, maxsplit);
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	6747
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	6748 Py_DECREF(s);
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	6749 Py_XDECREF(sep);
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	6750 return result;
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	6751 }
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	6752
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	6753 PyDoc_STRVAR(rsplit__doc__,
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	6754 "S.rsplit([sep [,maxsplit]]) -> list of strings\n\
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	6755 \n\
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	6756 Return a list of the words in S, using sep as the\n\
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	6757 delimiter string, starting at the end of the string and\n\
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	6758 working to the front. If maxsplit is given, at most maxsplit\n\
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	6759 splits are done. If sep is not specified, any whitespace string\n\
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	6760 is a separator.");
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	6761
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	6762 static PyObject*
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	6763 unicode_rsplit(PyUnicodeObject self, PyObject args)
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	6764 {
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	6765 PyObject *substring = Py_None;
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	6766 Py_ssize_t maxcount = -1;
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	6767
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	6768 if (!PyArg_ParseTuple(args, "\|On:rsplit", &substring, &maxcount))
31199 504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	6769 return NULL;
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	6770
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	6771 if (substring == Py_None)
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	6772 return rsplit(self, NULL, maxcount);
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	6773 else if (PyUnicode_Check(substring))
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	6774 return rsplit(self, (PyUnicodeObject *)substring, maxcount);
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	6775 else
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	6776 return PyUnicode_RSplit((PyObject *)self, substring, maxcount);
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	6777 }
504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	6778
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	6779 PyDoc_STRVAR(splitlines__doc__,
12369 68e48bb94071 Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12329 diff changeset	6780 "S.splitlines([keepends]]) -> list of strings\n\
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6781 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6782 Return a list of the lines in S, breaking at line boundaries.\n\
12369 68e48bb94071 Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12329 diff changeset	6783 Line breaks are not included in the resulting list unless keepends\n\
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	6784 is given and true.");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6785
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6786 static PyObject*
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6787 unicode_splitlines(PyUnicodeObject self, PyObject args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6788 {
12369 68e48bb94071 Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12329 diff changeset	6789 int keepends = 0;
68e48bb94071 Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12329 diff changeset	6790
68e48bb94071 Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12329 diff changeset	6791 if (!PyArg_ParseTuple(args, "\|i:splitlines", &keepends))
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6792 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6793
12369 68e48bb94071 Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12329 diff changeset	6794 return PyUnicode_Splitlines((PyObject *)self, keepends);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6795 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6796
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6797 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6798 PyObject unicode_str(PyUnicodeObject self)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6799 {
12645 c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	6800 return PyUnicode_AsEncodedString((PyObject *)self, NULL, NULL);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6801 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6802
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	6803 PyDoc_STRVAR(swapcase__doc__,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6804 "S.swapcase() -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6805 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6806 Return a copy of S with uppercase characters converted to lowercase\n\
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	6807 and vice versa.");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6808
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6809 static PyObject*
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	6810 unicode_swapcase(PyUnicodeObject *self)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6811 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6812 return fixup(self, fixswapcase);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6813 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6814
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	6815 PyDoc_STRVAR(translate__doc__,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6816 "S.translate(table) -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6817 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6818 Return a copy of the string S, where all characters have been mapped\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6819 through the given translation table, which must be a mapping of\n\
25120 953ebe0c0c25 Change the unicode.translate docstring to document that Walter Dörwald <walter@livinglogic.de> parents: 25091 diff changeset	6820 Unicode ordinals to Unicode ordinals, Unicode strings or None.\n\
953ebe0c0c25 Change the unicode.translate docstring to document that Walter Dörwald <walter@livinglogic.de> parents: 25091 diff changeset	6821 Unmapped characters are left untouched. Characters mapped to None\n\
953ebe0c0c25 Change the unicode.translate docstring to document that Walter Dörwald <walter@livinglogic.de> parents: 25091 diff changeset	6822 are deleted.");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6823
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6824 static PyObject*
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	6825 unicode_translate(PyUnicodeObject self, PyObject table)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6826 {
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	6827 return PyUnicode_TranslateCharmap(self->str,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6828 self->length,
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	6829 table,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6830 "ignore");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6831 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6832
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	6833 PyDoc_STRVAR(upper__doc__,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6834 "S.upper() -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6835 \n\
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	6836 Return a copy of S converted to uppercase.");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6837
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6838 static PyObject*
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	6839 unicode_upper(PyUnicodeObject *self)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6840 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6841 return fixup(self, fixupper);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6842 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6843
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	6844 PyDoc_STRVAR(zfill__doc__,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6845 "S.zfill(width) -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6846 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6847 Pad a numeric string x with zeros on the left, to fill a field\n\
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	6848 of the specified width. The string x is never truncated.");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6849
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6850 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6851 unicode_zfill(PyUnicodeObject self, PyObject args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6852 {
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	6853 Py_ssize_t fill;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6854 PyUnicodeObject *u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6855
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	6856 Py_ssize_t width;
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	6857 if (!PyArg_ParseTuple(args, "n:zfill", &width))
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6858 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6859
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6860 if (self->length >= width) {
23051 19ea00feda33 Return the orginal string only if it's a real str or unicode Walter Dörwald <walter@livinglogic.de> parents: 23045 diff changeset	6861 if (PyUnicode_CheckExact(self)) {
19ea00feda33 Return the orginal string only if it's a real str or unicode Walter Dörwald <walter@livinglogic.de> parents: 23045 diff changeset	6862 Py_INCREF(self);
19ea00feda33 Return the orginal string only if it's a real str or unicode Walter Dörwald <walter@livinglogic.de> parents: 23045 diff changeset	6863 return (PyObject*) self;
19ea00feda33 Return the orginal string only if it's a real str or unicode Walter Dörwald <walter@livinglogic.de> parents: 23045 diff changeset	6864 }
19ea00feda33 Return the orginal string only if it's a real str or unicode Walter Dörwald <walter@livinglogic.de> parents: 23045 diff changeset	6865 else
19ea00feda33 Return the orginal string only if it's a real str or unicode Walter Dörwald <walter@livinglogic.de> parents: 23045 diff changeset	6866 return PyUnicode_FromUnicode(
19ea00feda33 Return the orginal string only if it's a real str or unicode Walter Dörwald <walter@livinglogic.de> parents: 23045 diff changeset	6867 PyUnicode_AS_UNICODE(self),
19ea00feda33 Return the orginal string only if it's a real str or unicode Walter Dörwald <walter@livinglogic.de> parents: 23045 diff changeset	6868 PyUnicode_GET_SIZE(self)
19ea00feda33 Return the orginal string only if it's a real str or unicode Walter Dörwald <walter@livinglogic.de> parents: 23045 diff changeset	6869 );
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6870 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6871
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6872 fill = width - self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6873
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6874 u = pad(self, fill, 0, '0');
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6875
23045 8b1268bc371a Apply the second version of SF patch http://www.python.org/sf/536241 Walter Dörwald <walter@livinglogic.de> parents: 22981 diff changeset	6876 if (u == NULL)
8b1268bc371a Apply the second version of SF patch http://www.python.org/sf/536241 Walter Dörwald <walter@livinglogic.de> parents: 22981 diff changeset	6877 return NULL;
8b1268bc371a Apply the second version of SF patch http://www.python.org/sf/536241 Walter Dörwald <walter@livinglogic.de> parents: 22981 diff changeset	6878
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6879 if (u->str[fill] == '+' \|\| u->str[fill] == '-') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6880 /* move sign to beginning of string */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6881 u->str[0] = u->str[fill];
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6882 u->str[fill] = '0';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6883 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6884
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6885 return (PyObject*) u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6886 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6887
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6888 #if 0
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6889 static PyObject*
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	6890 unicode_freelistsize(PyUnicodeObject *self)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6891 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6892 return PyInt_FromLong(unicode_freelist_size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6893 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6894 #endif
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6895
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	6896 PyDoc_STRVAR(startswith__doc__,
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	6897 "S.startswith(prefix[, start[, end]]) -> bool\n\
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6898 \n\
28559 aff080f2efb4 Reformat a few docstrings that caused line wraps in help() output. Guido van Rossum <guido@python.org> parents: 28494 diff changeset	6899 Return True if S starts with the specified prefix, False otherwise.\n\
aff080f2efb4 Reformat a few docstrings that caused line wraps in help() output. Guido van Rossum <guido@python.org> parents: 28494 diff changeset	6900 With optional start, test S beginning at that position.\n\
38697 1cafbcd7f55c RFE #1491485: str/unicode.endswith()/startswith() now accept a tuple as first argument. Georg Brandl <georg@python.org> parents: 38587 diff changeset	6901 With optional end, stop comparing S at that position.\n\
1cafbcd7f55c RFE #1491485: str/unicode.endswith()/startswith() now accept a tuple as first argument. Georg Brandl <georg@python.org> parents: 38587 diff changeset	6902 prefix can also be a tuple of strings to try.");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6903
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6904 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6905 unicode_startswith(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6906 PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6907 {
38697 1cafbcd7f55c RFE #1491485: str/unicode.endswith()/startswith() now accept a tuple as first argument. Georg Brandl <georg@python.org> parents: 38587 diff changeset	6908 PyObject *subobj;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6909 PyUnicodeObject *substring;
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	6910 Py_ssize_t start = 0;
37662 0678ce9fb6b2 Change more ints to Py_ssize_t. Martin v. Löwis <martin@v.loewis.de> parents: 37661 diff changeset	6911 Py_ssize_t end = PY_SSIZE_T_MAX;
38697 1cafbcd7f55c RFE #1491485: str/unicode.endswith()/startswith() now accept a tuple as first argument. Georg Brandl <georg@python.org> parents: 38587 diff changeset	6912 int result;
1cafbcd7f55c RFE #1491485: str/unicode.endswith()/startswith() now accept a tuple as first argument. Georg Brandl <georg@python.org> parents: 38587 diff changeset	6913
1cafbcd7f55c RFE #1491485: str/unicode.endswith()/startswith() now accept a tuple as first argument. Georg Brandl <georg@python.org> parents: 38587 diff changeset	6914 if (!PyArg_ParseTuple(args, "O\|O&O&:startswith", &subobj,
12629 3efeaa7521e4 Trent Mick: Guido van Rossum <guido@python.org> parents: 12582 diff changeset	6915 _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end))
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6916 return NULL;
38697 1cafbcd7f55c RFE #1491485: str/unicode.endswith()/startswith() now accept a tuple as first argument. Georg Brandl <georg@python.org> parents: 38587 diff changeset	6917 if (PyTuple_Check(subobj)) {
1cafbcd7f55c RFE #1491485: str/unicode.endswith()/startswith() now accept a tuple as first argument. Georg Brandl <georg@python.org> parents: 38587 diff changeset	6918 Py_ssize_t i;
1cafbcd7f55c RFE #1491485: str/unicode.endswith()/startswith() now accept a tuple as first argument. Georg Brandl <georg@python.org> parents: 38587 diff changeset	6919 for (i = 0; i < PyTuple_GET_SIZE(subobj); i++) {
1cafbcd7f55c RFE #1491485: str/unicode.endswith()/startswith() now accept a tuple as first argument. Georg Brandl <georg@python.org> parents: 38587 diff changeset	6920 substring = (PyUnicodeObject *)PyUnicode_FromObject(
1cafbcd7f55c RFE #1491485: str/unicode.endswith()/startswith() now accept a tuple as first argument. Georg Brandl <georg@python.org> parents: 38587 diff changeset	6921 PyTuple_GET_ITEM(subobj, i));
1cafbcd7f55c RFE #1491485: str/unicode.endswith()/startswith() now accept a tuple as first argument. Georg Brandl <georg@python.org> parents: 38587 diff changeset	6922 if (substring == NULL)
1cafbcd7f55c RFE #1491485: str/unicode.endswith()/startswith() now accept a tuple as first argument. Georg Brandl <georg@python.org> parents: 38587 diff changeset	6923 return NULL;
1cafbcd7f55c RFE #1491485: str/unicode.endswith()/startswith() now accept a tuple as first argument. Georg Brandl <georg@python.org> parents: 38587 diff changeset	6924 result = tailmatch(self, substring, start, end, -1);
1cafbcd7f55c RFE #1491485: str/unicode.endswith()/startswith() now accept a tuple as first argument. Georg Brandl <georg@python.org> parents: 38587 diff changeset	6925 Py_DECREF(substring);
1cafbcd7f55c RFE #1491485: str/unicode.endswith()/startswith() now accept a tuple as first argument. Georg Brandl <georg@python.org> parents: 38587 diff changeset	6926 if (result) {
1cafbcd7f55c RFE #1491485: str/unicode.endswith()/startswith() now accept a tuple as first argument. Georg Brandl <georg@python.org> parents: 38587 diff changeset	6927 Py_RETURN_TRUE;
1cafbcd7f55c RFE #1491485: str/unicode.endswith()/startswith() now accept a tuple as first argument. Georg Brandl <georg@python.org> parents: 38587 diff changeset	6928 }
1cafbcd7f55c RFE #1491485: str/unicode.endswith()/startswith() now accept a tuple as first argument. Georg Brandl <georg@python.org> parents: 38587 diff changeset	6929 }
1cafbcd7f55c RFE #1491485: str/unicode.endswith()/startswith() now accept a tuple as first argument. Georg Brandl <georg@python.org> parents: 38587 diff changeset	6930 /* nothing matched */
1cafbcd7f55c RFE #1491485: str/unicode.endswith()/startswith() now accept a tuple as first argument. Georg Brandl <georg@python.org> parents: 38587 diff changeset	6931 Py_RETURN_FALSE;
1cafbcd7f55c RFE #1491485: str/unicode.endswith()/startswith() now accept a tuple as first argument. Georg Brandl <georg@python.org> parents: 38587 diff changeset	6932 }
1cafbcd7f55c RFE #1491485: str/unicode.endswith()/startswith() now accept a tuple as first argument. Georg Brandl <georg@python.org> parents: 38587 diff changeset	6933 substring = (PyUnicodeObject *)PyUnicode_FromObject(subobj);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6934 if (substring == NULL)
38697 1cafbcd7f55c RFE #1491485: str/unicode.endswith()/startswith() now accept a tuple as first argument. Georg Brandl <georg@python.org> parents: 38587 diff changeset	6935 return NULL;
1cafbcd7f55c RFE #1491485: str/unicode.endswith()/startswith() now accept a tuple as first argument. Georg Brandl <georg@python.org> parents: 38587 diff changeset	6936 result = tailmatch(self, substring, start, end, -1);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6937 Py_DECREF(substring);
38697 1cafbcd7f55c RFE #1491485: str/unicode.endswith()/startswith() now accept a tuple as first argument. Georg Brandl <georg@python.org> parents: 38587 diff changeset	6938 return PyBool_FromLong(result);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6939 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6940
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6941
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	6942 PyDoc_STRVAR(endswith__doc__,
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	6943 "S.endswith(suffix[, start[, end]]) -> bool\n\
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6944 \n\
28559 aff080f2efb4 Reformat a few docstrings that caused line wraps in help() output. Guido van Rossum <guido@python.org> parents: 28494 diff changeset	6945 Return True if S ends with the specified suffix, False otherwise.\n\
aff080f2efb4 Reformat a few docstrings that caused line wraps in help() output. Guido van Rossum <guido@python.org> parents: 28494 diff changeset	6946 With optional start, test S beginning at that position.\n\
38697 1cafbcd7f55c RFE #1491485: str/unicode.endswith()/startswith() now accept a tuple as first argument. Georg Brandl <georg@python.org> parents: 38587 diff changeset	6947 With optional end, stop comparing S at that position.\n\
1cafbcd7f55c RFE #1491485: str/unicode.endswith()/startswith() now accept a tuple as first argument. Georg Brandl <georg@python.org> parents: 38587 diff changeset	6948 suffix can also be a tuple of strings to try.");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6949
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6950 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6951 unicode_endswith(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6952 PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6953 {
38697 1cafbcd7f55c RFE #1491485: str/unicode.endswith()/startswith() now accept a tuple as first argument. Georg Brandl <georg@python.org> parents: 38587 diff changeset	6954 PyObject *subobj;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6955 PyUnicodeObject *substring;
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	6956 Py_ssize_t start = 0;
37662 0678ce9fb6b2 Change more ints to Py_ssize_t. Martin v. Löwis <martin@v.loewis.de> parents: 37661 diff changeset	6957 Py_ssize_t end = PY_SSIZE_T_MAX;
38697 1cafbcd7f55c RFE #1491485: str/unicode.endswith()/startswith() now accept a tuple as first argument. Georg Brandl <georg@python.org> parents: 38587 diff changeset	6958 int result;
1cafbcd7f55c RFE #1491485: str/unicode.endswith()/startswith() now accept a tuple as first argument. Georg Brandl <georg@python.org> parents: 38587 diff changeset	6959
1cafbcd7f55c RFE #1491485: str/unicode.endswith()/startswith() now accept a tuple as first argument. Georg Brandl <georg@python.org> parents: 38587 diff changeset	6960 if (!PyArg_ParseTuple(args, "O\|O&O&:endswith", &subobj,
1cafbcd7f55c RFE #1491485: str/unicode.endswith()/startswith() now accept a tuple as first argument. Georg Brandl <georg@python.org> parents: 38587 diff changeset	6961 _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end))
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6962 return NULL;
38697 1cafbcd7f55c RFE #1491485: str/unicode.endswith()/startswith() now accept a tuple as first argument. Georg Brandl <georg@python.org> parents: 38587 diff changeset	6963 if (PyTuple_Check(subobj)) {
1cafbcd7f55c RFE #1491485: str/unicode.endswith()/startswith() now accept a tuple as first argument. Georg Brandl <georg@python.org> parents: 38587 diff changeset	6964 Py_ssize_t i;
1cafbcd7f55c RFE #1491485: str/unicode.endswith()/startswith() now accept a tuple as first argument. Georg Brandl <georg@python.org> parents: 38587 diff changeset	6965 for (i = 0; i < PyTuple_GET_SIZE(subobj); i++) {
1cafbcd7f55c RFE #1491485: str/unicode.endswith()/startswith() now accept a tuple as first argument. Georg Brandl <georg@python.org> parents: 38587 diff changeset	6966 substring = (PyUnicodeObject *)PyUnicode_FromObject(
1cafbcd7f55c RFE #1491485: str/unicode.endswith()/startswith() now accept a tuple as first argument. Georg Brandl <georg@python.org> parents: 38587 diff changeset	6967 PyTuple_GET_ITEM(subobj, i));
1cafbcd7f55c RFE #1491485: str/unicode.endswith()/startswith() now accept a tuple as first argument. Georg Brandl <georg@python.org> parents: 38587 diff changeset	6968 if (substring == NULL)
1cafbcd7f55c RFE #1491485: str/unicode.endswith()/startswith() now accept a tuple as first argument. Georg Brandl <georg@python.org> parents: 38587 diff changeset	6969 return NULL;
1cafbcd7f55c RFE #1491485: str/unicode.endswith()/startswith() now accept a tuple as first argument. Georg Brandl <georg@python.org> parents: 38587 diff changeset	6970 result = tailmatch(self, substring, start, end, +1);
1cafbcd7f55c RFE #1491485: str/unicode.endswith()/startswith() now accept a tuple as first argument. Georg Brandl <georg@python.org> parents: 38587 diff changeset	6971 Py_DECREF(substring);
1cafbcd7f55c RFE #1491485: str/unicode.endswith()/startswith() now accept a tuple as first argument. Georg Brandl <georg@python.org> parents: 38587 diff changeset	6972 if (result) {
1cafbcd7f55c RFE #1491485: str/unicode.endswith()/startswith() now accept a tuple as first argument. Georg Brandl <georg@python.org> parents: 38587 diff changeset	6973 Py_RETURN_TRUE;
1cafbcd7f55c RFE #1491485: str/unicode.endswith()/startswith() now accept a tuple as first argument. Georg Brandl <georg@python.org> parents: 38587 diff changeset	6974 }
1cafbcd7f55c RFE #1491485: str/unicode.endswith()/startswith() now accept a tuple as first argument. Georg Brandl <georg@python.org> parents: 38587 diff changeset	6975 }
1cafbcd7f55c RFE #1491485: str/unicode.endswith()/startswith() now accept a tuple as first argument. Georg Brandl <georg@python.org> parents: 38587 diff changeset	6976 Py_RETURN_FALSE;
1cafbcd7f55c RFE #1491485: str/unicode.endswith()/startswith() now accept a tuple as first argument. Georg Brandl <georg@python.org> parents: 38587 diff changeset	6977 }
1cafbcd7f55c RFE #1491485: str/unicode.endswith()/startswith() now accept a tuple as first argument. Georg Brandl <georg@python.org> parents: 38587 diff changeset	6978 substring = (PyUnicodeObject *)PyUnicode_FromObject(subobj);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6979 if (substring == NULL)
38697 1cafbcd7f55c RFE #1491485: str/unicode.endswith()/startswith() now accept a tuple as first argument. Georg Brandl <georg@python.org> parents: 38587 diff changeset	6980 return NULL;
1cafbcd7f55c RFE #1491485: str/unicode.endswith()/startswith() now accept a tuple as first argument. Georg Brandl <georg@python.org> parents: 38587 diff changeset	6981
1cafbcd7f55c RFE #1491485: str/unicode.endswith()/startswith() now accept a tuple as first argument. Georg Brandl <georg@python.org> parents: 38587 diff changeset	6982 result = tailmatch(self, substring, start, end, +1);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6983 Py_DECREF(substring);
38697 1cafbcd7f55c RFE #1491485: str/unicode.endswith()/startswith() now accept a tuple as first argument. Georg Brandl <georg@python.org> parents: 38587 diff changeset	6984 return PyBool_FromLong(result);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6985 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6986
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6987
27521 17bfd5dbc0f1 Implement appropriate __getnewargs__ for all immutable subclassable builtin Guido van Rossum <guido@python.org> parents: 27175 diff changeset	6988
17bfd5dbc0f1 Implement appropriate __getnewargs__ for all immutable subclassable builtin Guido van Rossum <guido@python.org> parents: 27175 diff changeset	6989 static PyObject *
17bfd5dbc0f1 Implement appropriate __getnewargs__ for all immutable subclassable builtin Guido van Rossum <guido@python.org> parents: 27175 diff changeset	6990 unicode_getnewargs(PyUnicodeObject *v)
17bfd5dbc0f1 Implement appropriate __getnewargs__ for all immutable subclassable builtin Guido van Rossum <guido@python.org> parents: 27175 diff changeset	6991 {
17bfd5dbc0f1 Implement appropriate __getnewargs__ for all immutable subclassable builtin Guido van Rossum <guido@python.org> parents: 27175 diff changeset	6992 return Py_BuildValue("(u#)", v->str, v->length);
17bfd5dbc0f1 Implement appropriate __getnewargs__ for all immutable subclassable builtin Guido van Rossum <guido@python.org> parents: 27175 diff changeset	6993 }
17bfd5dbc0f1 Implement appropriate __getnewargs__ for all immutable subclassable builtin Guido van Rossum <guido@python.org> parents: 27175 diff changeset	6994
17bfd5dbc0f1 Implement appropriate __getnewargs__ for all immutable subclassable builtin Guido van Rossum <guido@python.org> parents: 27175 diff changeset	6995
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6996 static PyMethodDef unicode_methods[] = {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6997
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6998 /* Order is according to common usage: often used methods should
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6999 appear first, since lookup is done sequentially. */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7000
37306 ce56c5e3cb20 That one was a mistake. Georg Brandl <georg@python.org> parents: 37304 diff changeset	7001 {"encode", (PyCFunction) unicode_encode, METH_VARARGS, encode__doc__},
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	7002 {"replace", (PyCFunction) unicode_replace, METH_VARARGS, replace__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	7003 {"split", (PyCFunction) unicode_split, METH_VARARGS, split__doc__},
31199 504943c89193 Add rsplit method for str and unicode builtin types. Hye-Shik Chang <hyeshik@gmail.com> parents: 31042 diff changeset	7004 {"rsplit", (PyCFunction) unicode_rsplit, METH_VARARGS, rsplit__doc__},
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	7005 {"join", (PyCFunction) unicode_join, METH_O, join__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	7006 {"capitalize", (PyCFunction) unicode_capitalize, METH_NOARGS, capitalize__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	7007 {"title", (PyCFunction) unicode_title, METH_NOARGS, title__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	7008 {"center", (PyCFunction) unicode_center, METH_VARARGS, center__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	7009 {"count", (PyCFunction) unicode_count, METH_VARARGS, count__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	7010 {"expandtabs", (PyCFunction) unicode_expandtabs, METH_VARARGS, expandtabs__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	7011 {"find", (PyCFunction) unicode_find, METH_VARARGS, find__doc__},
38338 958f91684209 needforspeed: use METH_O for argument handling, which made partition some Fredrik Lundh <fredrik@pythonware.com> parents: 38335 diff changeset	7012 {"partition", (PyCFunction) unicode_partition, METH_O, partition__doc__},
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	7013 {"index", (PyCFunction) unicode_index, METH_VARARGS, index__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	7014 {"ljust", (PyCFunction) unicode_ljust, METH_VARARGS, ljust__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	7015 {"lower", (PyCFunction) unicode_lower, METH_NOARGS, lower__doc__},
23225 e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	7016 {"lstrip", (PyCFunction) unicode_lstrip, METH_VARARGS, lstrip__doc__},
32551 f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode() Marc-André Lemburg <mal@egenix.com> parents: 32363 diff changeset	7017 {"decode", (PyCFunction) unicode_decode, METH_VARARGS, decode__doc__},
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	7018 /* {"maketrans", (PyCFunction) unicode_maketrans, METH_VARARGS, maketrans__doc__}, */
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	7019 {"rfind", (PyCFunction) unicode_rfind, METH_VARARGS, rfind__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	7020 {"rindex", (PyCFunction) unicode_rindex, METH_VARARGS, rindex__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	7021 {"rjust", (PyCFunction) unicode_rjust, METH_VARARGS, rjust__doc__},
23225 e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	7022 {"rstrip", (PyCFunction) unicode_rstrip, METH_VARARGS, rstrip__doc__},
38390 76807902a2f5 needforspeed: added rpartition implementation Fredrik Lundh <fredrik@pythonware.com> parents: 38383 diff changeset	7023 {"rpartition", (PyCFunction) unicode_rpartition, METH_O, rpartition__doc__},
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	7024 {"splitlines", (PyCFunction) unicode_splitlines, METH_VARARGS, splitlines__doc__},
23225 e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	7025 {"strip", (PyCFunction) unicode_strip, METH_VARARGS, strip__doc__},
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	7026 {"swapcase", (PyCFunction) unicode_swapcase, METH_NOARGS, swapcase__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	7027 {"translate", (PyCFunction) unicode_translate, METH_O, translate__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	7028 {"upper", (PyCFunction) unicode_upper, METH_NOARGS, upper__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	7029 {"startswith", (PyCFunction) unicode_startswith, METH_VARARGS, startswith__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	7030 {"endswith", (PyCFunction) unicode_endswith, METH_VARARGS, endswith__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	7031 {"islower", (PyCFunction) unicode_islower, METH_NOARGS, islower__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	7032 {"isupper", (PyCFunction) unicode_isupper, METH_NOARGS, isupper__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	7033 {"istitle", (PyCFunction) unicode_istitle, METH_NOARGS, istitle__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	7034 {"isspace", (PyCFunction) unicode_isspace, METH_NOARGS, isspace__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	7035 {"isdecimal", (PyCFunction) unicode_isdecimal, METH_NOARGS, isdecimal__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	7036 {"isdigit", (PyCFunction) unicode_isdigit, METH_NOARGS, isdigit__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	7037 {"isnumeric", (PyCFunction) unicode_isnumeric, METH_NOARGS, isnumeric__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	7038 {"isalpha", (PyCFunction) unicode_isalpha, METH_NOARGS, isalpha__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	7039 {"isalnum", (PyCFunction) unicode_isalnum, METH_NOARGS, isalnum__doc__},
23045 8b1268bc371a Apply the second version of SF patch http://www.python.org/sf/536241 Walter Dörwald <walter@livinglogic.de> parents: 22981 diff changeset	7040 {"zfill", (PyCFunction) unicode_zfill, METH_VARARGS, zfill__doc__},
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7041 #if 0
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	7042 {"capwords", (PyCFunction) unicode_capwords, METH_NOARGS, capwords__doc__},
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7043 #endif
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7044
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7045 #if 0
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7046 /* This one is just used for debugging the implementation. */
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	7047 {"freelistsize", (PyCFunction) unicode_freelistsize, METH_NOARGS},
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7048 #endif
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7049
27521 17bfd5dbc0f1 Implement appropriate __getnewargs__ for all immutable subclassable builtin Guido van Rossum <guido@python.org> parents: 27175 diff changeset	7050 {"__getnewargs__", (PyCFunction)unicode_getnewargs, METH_NOARGS},
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7051 {NULL, NULL}
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7052 };
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7053
26274 958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 26164 diff changeset	7054 static PyObject *
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 26164 diff changeset	7055 unicode_mod(PyObject v, PyObject w)
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 26164 diff changeset	7056 {
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 26164 diff changeset	7057 if (!PyUnicode_Check(v)) {
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 26164 diff changeset	7058 Py_INCREF(Py_NotImplemented);
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 26164 diff changeset	7059 return Py_NotImplemented;
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 26164 diff changeset	7060 }
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 26164 diff changeset	7061 return PyUnicode_Format(v, w);
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 26164 diff changeset	7062 }
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 26164 diff changeset	7063
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 26164 diff changeset	7064 static PyNumberMethods unicode_as_number = {
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 26164 diff changeset	7065 0, /nb_add/
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 26164 diff changeset	7066 0, /nb_subtract/
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 26164 diff changeset	7067 0, /nb_multiply/
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 26164 diff changeset	7068 0, /nb_divide/
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 26164 diff changeset	7069 unicode_mod, /nb_remainder/
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 26164 diff changeset	7070 };
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 26164 diff changeset	7071
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7072 static PySequenceMethods unicode_as_sequence = {
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	7073 (lenfunc) unicode_length, /* sq_length */
37304 79a0f080f898 Remove unnecessary casts in type object initializers. Georg Brandl <georg@python.org> parents: 36945 diff changeset	7074 PyUnicode_Concat, /* sq_concat */
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	7075 (ssizeargfunc) unicode_repeat, /* sq_repeat */
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	7076 (ssizeargfunc) unicode_getitem, /* sq_item */
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	7077 (ssizessizeargfunc) unicode_slice, /* sq_slice */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7078 0, /* sq_ass_item */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7079 0, /* sq_ass_slice */
37304 79a0f080f898 Remove unnecessary casts in type object initializers. Georg Brandl <georg@python.org> parents: 36945 diff changeset	7080 PyUnicode_Contains, /* sq_contains */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7081 };
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7082
23778 245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	7083 static PyObject*
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	7084 unicode_subscript(PyUnicodeObject* self, PyObject* item)
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	7085 {
39491 2312e178b34d Correct an accidentally removed previous patch. Marc-André Lemburg <mal@egenix.com> parents: 39487 diff changeset	7086 if (PyIndex_Check(item)) {
2312e178b34d Correct an accidentally removed previous patch. Marc-André Lemburg <mal@egenix.com> parents: 39487 diff changeset	7087 Py_ssize_t i = PyNumber_AsSsize_t(item, PyExc_IndexError);
23778 245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	7088 if (i == -1 && PyErr_Occurred())
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	7089 return NULL;
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	7090 if (i < 0)
36203 44cfca6136c6 Stop maintaining the buildno file. Martin v. Löwis <martin@v.loewis.de> parents: 36056 diff changeset	7091 i += PyUnicode_GET_SIZE(self);
23778 245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	7092 return unicode_getitem(self, i);
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	7093 } else if (PySlice_Check(item)) {
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	7094 Py_ssize_t start, stop, step, slicelength, cur, i;
23778 245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	7095 Py_UNICODE* source_buf;
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	7096 Py_UNICODE* result_buf;
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	7097 PyObject* result;
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	7098
36203 44cfca6136c6 Stop maintaining the buildno file. Martin v. Löwis <martin@v.loewis.de> parents: 36056 diff changeset	7099 if (PySlice_GetIndicesEx((PySliceObject*)item, PyUnicode_GET_SIZE(self),
23778 245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	7100 &start, &stop, &step, &slicelength) < 0) {
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	7101 return NULL;
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	7102 }
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	7103
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	7104 if (slicelength <= 0) {
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	7105 return PyUnicode_FromUnicode(NULL, 0);
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	7106 } else {
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	7107 source_buf = PyUnicode_AS_UNICODE((PyObject*)self);
37596 1b078493d538 More low-hanging fruit. Still need to re-arrange some code (or find a better Anthony Baxter <anthonybaxter@gmail.com> parents: 37306 diff changeset	7108 result_buf = (Py_UNICODE )PyMem_MALLOC(slicelength
1b078493d538 More low-hanging fruit. Still need to re-arrange some code (or find a better Anthony Baxter <anthonybaxter@gmail.com> parents: 37306 diff changeset	7109 sizeof(Py_UNICODE));
36203 44cfca6136c6 Stop maintaining the buildno file. Martin v. Löwis <martin@v.loewis.de> parents: 36056 diff changeset	7110
44cfca6136c6 Stop maintaining the buildno file. Martin v. Löwis <martin@v.loewis.de> parents: 36056 diff changeset	7111 if (result_buf == NULL)
44cfca6136c6 Stop maintaining the buildno file. Martin v. Löwis <martin@v.loewis.de> parents: 36056 diff changeset	7112 return PyErr_NoMemory();
23778 245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	7113
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	7114 for (cur = start, i = 0; i < slicelength; cur += step, i++) {
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	7115 result_buf[i] = source_buf[cur];
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	7116 }
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	7117
23778 245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	7118 result = PyUnicode_FromUnicode(result_buf, slicelength);
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	7119 PyMem_FREE(result_buf);
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	7120 return result;
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	7121 }
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	7122 } else {
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	7123 PyErr_SetString(PyExc_TypeError, "string indices must be integers");
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	7124 return NULL;
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	7125 }
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	7126 }
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	7127
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	7128 static PyMappingMethods unicode_as_mapping = {
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	7129 (lenfunc)unicode_length, /* mp_length */
23778 245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	7130 (binaryfunc)unicode_subscript, /* mp_subscript */
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	7131 (objobjargproc)0, /* mp_ass_subscript */
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	7132 };
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	7133
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	7134 static Py_ssize_t
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7135 unicode_buffer_getreadbuf(PyUnicodeObject *self,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	7136 Py_ssize_t index,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7137 const void **ptr)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7138 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7139 if (index != 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7140 PyErr_SetString(PyExc_SystemError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7141 "accessing non-existent unicode segment");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7142 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7143 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7144 ptr = (void ) self->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7145 return PyUnicode_GET_DATA_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7146 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7147
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	7148 static Py_ssize_t
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	7149 unicode_buffer_getwritebuf(PyUnicodeObject *self, Py_ssize_t index,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7150 const void **ptr)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7151 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7152 PyErr_SetString(PyExc_TypeError,
23837 359a57e96d5e Fix typo in exception message Neal Norwitz <nnorwitz@gmail.com> parents: 23832 diff changeset	7153 "cannot use unicode as modifiable buffer");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7154 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7155 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7156
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7157 static int
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7158 unicode_buffer_getsegcount(PyUnicodeObject *self,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	7159 Py_ssize_t *lenp)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7160 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7161 if (lenp)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7162 *lenp = PyUnicode_GET_DATA_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7163 return 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7164 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7165
36519 d6d415df0df0 Use Py_ssize_t for counts and sizes. Martin v. Löwis <martin@v.loewis.de> parents: 36510 diff changeset	7166 static Py_ssize_t
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7167 unicode_buffer_getcharbuf(PyUnicodeObject *self,
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	7168 Py_ssize_t index,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7169 const void **ptr)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7170 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7171 PyObject *str;
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	7172
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7173 if (index != 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7174 PyErr_SetString(PyExc_SystemError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7175 "accessing non-existent unicode segment");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7176 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7177 }
13837 2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	7178 str = _PyUnicode_AsDefaultEncodedString((PyObject *)self, NULL);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7179 if (str == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7180 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7181 ptr = (void ) PyString_AS_STRING(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7182 return PyString_GET_SIZE(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7183 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7184
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7185 /* Helpers for PyUnicode_Format() */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7186
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7187 static PyObject *
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	7188 getnextarg(PyObject args, Py_ssize_t arglen, Py_ssize_t p_argidx)
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	7189 {
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	7190 Py_ssize_t argidx = *p_argidx;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7191 if (argidx < arglen) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7192 (*p_argidx)++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7193 if (arglen < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7194 return args;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7195 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7196 return PyTuple_GetItem(args, argidx);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7197 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7198 PyErr_SetString(PyExc_TypeError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7199 "not enough arguments for format string");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7200 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7201 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7202
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7203 #define F_LJUST (1<<0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7204 #define F_SIGN (1<<1)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7205 #define F_BLANK (1<<2)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7206 #define F_ALT (1<<3)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7207 #define F_ZERO (1<<4)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7208
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	7209 static Py_ssize_t
36261 303dea847639 - Patch #1400181, fix unicode string formatting to not use the locale. Neal Norwitz <nnorwitz@gmail.com> parents: 36236 diff changeset	7210 strtounicode(Py_UNICODE buffer, const char charbuffer)
303dea847639 - Patch #1400181, fix unicode string formatting to not use the locale. Neal Norwitz <nnorwitz@gmail.com> parents: 36236 diff changeset	7211 {
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	7212 register Py_ssize_t i;
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	7213 Py_ssize_t len = strlen(charbuffer);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7214 for (i = len - 1; i >= 0; i--)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7215 buffer[i] = (Py_UNICODE) charbuffer[i];
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7216
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7217 return len;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7218 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7219
36261 303dea847639 - Patch #1400181, fix unicode string formatting to not use the locale. Neal Norwitz <nnorwitz@gmail.com> parents: 36236 diff changeset	7220 static int
303dea847639 - Patch #1400181, fix unicode string formatting to not use the locale. Neal Norwitz <nnorwitz@gmail.com> parents: 36236 diff changeset	7221 doubletounicode(Py_UNICODE buffer, size_t len, const char format, double x)
303dea847639 - Patch #1400181, fix unicode string formatting to not use the locale. Neal Norwitz <nnorwitz@gmail.com> parents: 36236 diff changeset	7222 {
36508 7322dccba224 doubletounicode(), longtounicode(): Tim Peters <tim.peters@gmail.com> parents: 36506 diff changeset	7223 Py_ssize_t result;
7322dccba224 doubletounicode(), longtounicode(): Tim Peters <tim.peters@gmail.com> parents: 36506 diff changeset	7224
36261 303dea847639 - Patch #1400181, fix unicode string formatting to not use the locale. Neal Norwitz <nnorwitz@gmail.com> parents: 36236 diff changeset	7225 PyOS_ascii_formatd((char *)buffer, len, format, x);
36508 7322dccba224 doubletounicode(), longtounicode(): Tim Peters <tim.peters@gmail.com> parents: 36506 diff changeset	7226 result = strtounicode(buffer, (char *)buffer);
7322dccba224 doubletounicode(), longtounicode(): Tim Peters <tim.peters@gmail.com> parents: 36506 diff changeset	7227 return Py_SAFE_DOWNCAST(result, Py_ssize_t, int);
36261 303dea847639 - Patch #1400181, fix unicode string formatting to not use the locale. Neal Norwitz <nnorwitz@gmail.com> parents: 36236 diff changeset	7228 }
303dea847639 - Patch #1400181, fix unicode string formatting to not use the locale. Neal Norwitz <nnorwitz@gmail.com> parents: 36236 diff changeset	7229
303dea847639 - Patch #1400181, fix unicode string formatting to not use the locale. Neal Norwitz <nnorwitz@gmail.com> parents: 36236 diff changeset	7230 static int
303dea847639 - Patch #1400181, fix unicode string formatting to not use the locale. Neal Norwitz <nnorwitz@gmail.com> parents: 36236 diff changeset	7231 longtounicode(Py_UNICODE buffer, size_t len, const char format, long x)
303dea847639 - Patch #1400181, fix unicode string formatting to not use the locale. Neal Norwitz <nnorwitz@gmail.com> parents: 36236 diff changeset	7232 {
36508 7322dccba224 doubletounicode(), longtounicode(): Tim Peters <tim.peters@gmail.com> parents: 36506 diff changeset	7233 Py_ssize_t result;
7322dccba224 doubletounicode(), longtounicode(): Tim Peters <tim.peters@gmail.com> parents: 36506 diff changeset	7234
36261 303dea847639 - Patch #1400181, fix unicode string formatting to not use the locale. Neal Norwitz <nnorwitz@gmail.com> parents: 36236 diff changeset	7235 PyOS_snprintf((char *)buffer, len, format, x);
36508 7322dccba224 doubletounicode(), longtounicode(): Tim Peters <tim.peters@gmail.com> parents: 36506 diff changeset	7236 result = strtounicode(buffer, (char *)buffer);
7322dccba224 doubletounicode(), longtounicode(): Tim Peters <tim.peters@gmail.com> parents: 36506 diff changeset	7237 return Py_SAFE_DOWNCAST(result, Py_ssize_t, int);
36261 303dea847639 - Patch #1400181, fix unicode string formatting to not use the locale. Neal Norwitz <nnorwitz@gmail.com> parents: 36236 diff changeset	7238 }
303dea847639 - Patch #1400181, fix unicode string formatting to not use the locale. Neal Norwitz <nnorwitz@gmail.com> parents: 36236 diff changeset	7239
24736 ff05eef3a6d1 Implement stage B0 of PEP 237: add warnings for operations that Guido van Rossum <guido@python.org> parents: 24703 diff changeset	7240 /* XXX To save some code duplication, formatfloat/long/int could have been
ff05eef3a6d1 Implement stage B0 of PEP 237: add warnings for operations that Guido van Rossum <guido@python.org> parents: 24703 diff changeset	7241 shared with stringobject.c, converting from 8-bit to Unicode after the
ff05eef3a6d1 Implement stage B0 of PEP 237: add warnings for operations that Guido van Rossum <guido@python.org> parents: 24703 diff changeset	7242 formatting is done. */
ff05eef3a6d1 Implement stage B0 of PEP 237: add warnings for operations that Guido van Rossum <guido@python.org> parents: 24703 diff changeset	7243
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7244 static int
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7245 formatfloat(Py_UNICODE *buf,
13196 18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	7246 size_t buflen,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7247 int flags,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7248 int prec,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7249 int type,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7250 PyObject *v)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7251 {
13196 18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	7252 /* fmt = '%#.' + `prec` + `type`
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	7253 worst case length = 3 + 10 (len of INT_MAX) + 1 = 14 (use 20)*/
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7254 char fmt[20];
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7255 double x;
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	7256
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7257 x = PyFloat_AsDouble(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7258 if (x == -1.0 && PyErr_Occurred())
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7259 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7260 if (prec < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7261 prec = 6;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7262 if (type == 'f' && (fabs(x) / 1e25) >= 1e25)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7263 type = 'g';
26825 e153961f961f Patch for bug #659709: bogus computation of float length Marc-André Lemburg <mal@egenix.com> parents: 26274 diff changeset	7264 /* Worst case length calc to ensure no buffer overrun:
e153961f961f Patch for bug #659709: bogus computation of float length Marc-André Lemburg <mal@egenix.com> parents: 26274 diff changeset	7265
e153961f961f Patch for bug #659709: bogus computation of float length Marc-André Lemburg <mal@egenix.com> parents: 26274 diff changeset	7266 'g' formats:
e153961f961f Patch for bug #659709: bogus computation of float length Marc-André Lemburg <mal@egenix.com> parents: 26274 diff changeset	7267 fmt = %#.<prec>g
e153961f961f Patch for bug #659709: bogus computation of float length Marc-André Lemburg <mal@egenix.com> parents: 26274 diff changeset	7268 buf = '-' + [0-9]*prec + '.' + 'e+' + (longest exp
e153961f961f Patch for bug #659709: bogus computation of float length Marc-André Lemburg <mal@egenix.com> parents: 26274 diff changeset	7269 for any double rep.)
e153961f961f Patch for bug #659709: bogus computation of float length Marc-André Lemburg <mal@egenix.com> parents: 26274 diff changeset	7270 len = 1 + prec + 1 + 2 + 5 = 9 + prec
e153961f961f Patch for bug #659709: bogus computation of float length Marc-André Lemburg <mal@egenix.com> parents: 26274 diff changeset	7271
e153961f961f Patch for bug #659709: bogus computation of float length Marc-André Lemburg <mal@egenix.com> parents: 26274 diff changeset	7272 'f' formats:
e153961f961f Patch for bug #659709: bogus computation of float length Marc-André Lemburg <mal@egenix.com> parents: 26274 diff changeset	7273 buf = '-' + [0-9]x + '.' + [0-9]prec (with x < 50)
e153961f961f Patch for bug #659709: bogus computation of float length Marc-André Lemburg <mal@egenix.com> parents: 26274 diff changeset	7274 len = 1 + 50 + 1 + prec = 52 + prec
e153961f961f Patch for bug #659709: bogus computation of float length Marc-André Lemburg <mal@egenix.com> parents: 26274 diff changeset	7275
e153961f961f Patch for bug #659709: bogus computation of float length Marc-André Lemburg <mal@egenix.com> parents: 26274 diff changeset	7276 If prec=0 the effective precision is 1 (the leading digit is
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	7277 always given), therefore increase the length by one.
26825 e153961f961f Patch for bug #659709: bogus computation of float length Marc-André Lemburg <mal@egenix.com> parents: 26274 diff changeset	7278
e153961f961f Patch for bug #659709: bogus computation of float length Marc-André Lemburg <mal@egenix.com> parents: 26274 diff changeset	7279 */
e153961f961f Patch for bug #659709: bogus computation of float length Marc-André Lemburg <mal@egenix.com> parents: 26274 diff changeset	7280 if ((type == 'g' && buflen <= (size_t)10 + (size_t)prec) \|\|
e153961f961f Patch for bug #659709: bogus computation of float length Marc-André Lemburg <mal@egenix.com> parents: 26274 diff changeset	7281 (type == 'f' && buflen <= (size_t)53 + (size_t)prec)) {
e153961f961f Patch for bug #659709: bogus computation of float length Marc-André Lemburg <mal@egenix.com> parents: 26274 diff changeset	7282 PyErr_SetString(PyExc_OverflowError,
e153961f961f Patch for bug #659709: bogus computation of float length Marc-André Lemburg <mal@egenix.com> parents: 26274 diff changeset	7283 "formatted float is too long (precision too large?)");
e153961f961f Patch for bug #659709: bogus computation of float length Marc-André Lemburg <mal@egenix.com> parents: 26274 diff changeset	7284 return -1;
e153961f961f Patch for bug #659709: bogus computation of float length Marc-André Lemburg <mal@egenix.com> parents: 26274 diff changeset	7285 }
21082 c8d775b9ff92 formatfloat(), formatint(): Conversion of sprintf() to PyOS_snprintf() Barry Warsaw <barry@python.org> parents: 21067 diff changeset	7286 PyOS_snprintf(fmt, sizeof(fmt), "%%%s.%d%c",
26825 e153961f961f Patch for bug #659709: bogus computation of float length Marc-André Lemburg <mal@egenix.com> parents: 26274 diff changeset	7287 (flags&F_ALT) ? "#" : "",
e153961f961f Patch for bug #659709: bogus computation of float length Marc-André Lemburg <mal@egenix.com> parents: 26274 diff changeset	7288 prec, type);
36261 303dea847639 - Patch #1400181, fix unicode string formatting to not use the locale. Neal Norwitz <nnorwitz@gmail.com> parents: 36236 diff changeset	7289 return doubletounicode(buf, buflen, fmt, x);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7290 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7291
14713 58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	7292 static PyObject*
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	7293 formatlong(PyObject *val, int flags, int prec, int type)
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	7294 {
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	7295 char *buf;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	7296 int i, len;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	7297 PyObject str; / temporary string object. */
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	7298 PyUnicodeObject *result;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	7299
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	7300 str = _PyString_FormatLong(val, flags, prec, type, &buf, &len);
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	7301 if (!str)
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	7302 return NULL;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	7303 result = _PyUnicode_New(len);
36877 e9a7a2784e17 SF #1444030: Fix several potential defects found by Coverity. Hye-Shik Chang <hyeshik@gmail.com> parents: 36675 diff changeset	7304 if (!result) {
e9a7a2784e17 SF #1444030: Fix several potential defects found by Coverity. Hye-Shik Chang <hyeshik@gmail.com> parents: 36675 diff changeset	7305 Py_DECREF(str);
e9a7a2784e17 SF #1444030: Fix several potential defects found by Coverity. Hye-Shik Chang <hyeshik@gmail.com> parents: 36675 diff changeset	7306 return NULL;
e9a7a2784e17 SF #1444030: Fix several potential defects found by Coverity. Hye-Shik Chang <hyeshik@gmail.com> parents: 36675 diff changeset	7307 }
14713 58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	7308 for (i = 0; i < len; i++)
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	7309 result->str[i] = buf[i];
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	7310 result->str[len] = 0;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	7311 Py_DECREF(str);
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	7312 return (PyObject*)result;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	7313 }
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	7314
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7315 static int
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7316 formatint(Py_UNICODE *buf,
13196 18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	7317 size_t buflen,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7318 int flags,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7319 int prec,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7320 int type,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7321 PyObject *v)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7322 {
13196 18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	7323 /* fmt = '%#.' + `prec` + 'l' + `type`
22087 5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	7324 * worst case length = 3 + 19 (worst len of INT_MAX on 64-bit machine)
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	7325 * + 1 + 1
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	7326 * = 24
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	7327 */
14713 58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	7328 char fmt[64]; /* plenty big enough! */
31042 9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions Guido van Rossum <guido@python.org> parents: 31021 diff changeset	7329 char *sign;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7330 long x;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7331
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7332 x = PyInt_AsLong(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7333 if (x == -1 && PyErr_Occurred())
22087 5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	7334 return -1;
31042 9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions Guido van Rossum <guido@python.org> parents: 31021 diff changeset	7335 if (x < 0 && type == 'u') {
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions Guido van Rossum <guido@python.org> parents: 31021 diff changeset	7336 type = 'd';
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions Guido van Rossum <guido@python.org> parents: 31021 diff changeset	7337 }
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions Guido van Rossum <guido@python.org> parents: 31021 diff changeset	7338 if (x < 0 && (type == 'x' \|\| type == 'X' \|\| type == 'o'))
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions Guido van Rossum <guido@python.org> parents: 31021 diff changeset	7339 sign = "-";
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions Guido van Rossum <guido@python.org> parents: 31021 diff changeset	7340 else
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions Guido van Rossum <guido@python.org> parents: 31021 diff changeset	7341 sign = "";
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7342 if (prec < 0)
22087 5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	7343 prec = 1;
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	7344
31042 9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions Guido van Rossum <guido@python.org> parents: 31021 diff changeset	7345 /* buf = '+'/'-'/'' + '0'/'0x'/'' + '[0-9]'*max(prec, len(x in octal))
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions Guido van Rossum <guido@python.org> parents: 31021 diff changeset	7346 * worst case buf = '-0x' + [0-9]*prec, where prec >= 11
17450 f74f5aafdc7e Bug 415514 reported that e.g. Tim Peters <tim.peters@gmail.com> parents: 17425 diff changeset	7347 */
31042 9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions Guido van Rossum <guido@python.org> parents: 31021 diff changeset	7348 if (buflen <= 14 \|\| buflen <= (size_t)3 + (size_t)prec) {
22087 5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	7349 PyErr_SetString(PyExc_OverflowError,
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	7350 "formatted integer is too long (precision too large?)");
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	7351 return -1;
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	7352 }
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	7353
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	7354 if ((flags & F_ALT) &&
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	7355 (type == 'x' \|\| type == 'X')) {
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	7356 /* When converting under %#x or %#X, there are a number
22087 5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	7357 * of issues that cause pain:
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	7358 * - when 0 is being converted, the C standard leaves off
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	7359 * the '0x' or '0X', which is inconsistent with other
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	7360 * %#x/%#X conversions and inconsistent with Python's
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	7361 * hex() function
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	7362 * - there are platforms that violate the standard and
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	7363 * convert 0 with the '0x' or '0X'
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	7364 * (Metrowerks, Compaq Tru64)
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	7365 * - there are platforms that give '0x' when converting
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	7366 * under %#X, but convert 0 in accordance with the
22087 5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	7367 * standard (OS/2 EMX)
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	7368 *
22087 5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	7369 * We can achieve the desired consistency by inserting our
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	7370 * own '0x' or '0X' prefix, and substituting %x/%X in place
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	7371 * of %#x/%#X.
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	7372 *
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	7373 * Note that this is the same approach as used in
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	7374 * formatint() in stringobject.c
22053 49197fd76362 OS/2 EMX port changes (Objects part of patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 21886 diff changeset	7375 */
31042 9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions Guido van Rossum <guido@python.org> parents: 31021 diff changeset	7376 PyOS_snprintf(fmt, sizeof(fmt), "%s0%c%%.%dl%c",
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions Guido van Rossum <guido@python.org> parents: 31021 diff changeset	7377 sign, type, prec, type);
22087 5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	7378 }
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	7379 else {
31042 9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions Guido van Rossum <guido@python.org> parents: 31021 diff changeset	7380 PyOS_snprintf(fmt, sizeof(fmt), "%s%%%s.%dl%c",
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions Guido van Rossum <guido@python.org> parents: 31021 diff changeset	7381 sign, (flags&F_ALT) ? "#" : "",
22087 5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	7382 prec, type);
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	7383 }
31042 9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions Guido van Rossum <guido@python.org> parents: 31021 diff changeset	7384 if (sign[0])
36261 303dea847639 - Patch #1400181, fix unicode string formatting to not use the locale. Neal Norwitz <nnorwitz@gmail.com> parents: 36236 diff changeset	7385 return longtounicode(buf, buflen, fmt, -x);
31042 9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions Guido van Rossum <guido@python.org> parents: 31021 diff changeset	7386 else
36261 303dea847639 - Patch #1400181, fix unicode string formatting to not use the locale. Neal Norwitz <nnorwitz@gmail.com> parents: 36236 diff changeset	7387 return longtounicode(buf, buflen, fmt, x);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7388 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7389
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7390 static int
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7391 formatchar(Py_UNICODE *buf,
13196 18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	7392 size_t buflen,
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	7393 PyObject *v)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7394 {
13196 18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	7395 /* presume that the buffer is at least 2 characters long */
12899 9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	7396 if (PyUnicode_Check(v)) {
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	7397 if (PyUnicode_GET_SIZE(v) != 1)
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	7398 goto onError;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7399 buf[0] = PyUnicode_AS_UNICODE(v)[0];
12899 9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	7400 }
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	7401
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	7402 else if (PyString_Check(v)) {
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	7403 if (PyString_GET_SIZE(v) != 1)
12899 9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	7404 goto onError;
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	7405 buf[0] = (Py_UNICODE)PyString_AS_STRING(v)[0];
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	7406 }
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7407
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7408 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7409 /* Integer input truncated to a character */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7410 long x;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7411 x = PyInt_AsLong(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7412 if (x == -1 && PyErr_Occurred())
12899 9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	7413 goto onError;
24737 eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	7414 #ifdef Py_UNICODE_WIDE
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	7415 if (x < 0 \|\| x > 0x10ffff) {
28494 32bea41fef68 Change formatchar(), so that u"%c" % 0xffffffff now raises Walter Dörwald <walter@livinglogic.de> parents: 28242 diff changeset	7416 PyErr_SetString(PyExc_OverflowError,
24737 eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	7417 "%c arg not in range(0x110000) "
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	7418 "(wide Python build)");
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	7419 return -1;
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	7420 }
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	7421 #else
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	7422 if (x < 0 \|\| x > 0xffff) {
28494 32bea41fef68 Change formatchar(), so that u"%c" % 0xffffffff now raises Walter Dörwald <walter@livinglogic.de> parents: 28242 diff changeset	7423 PyErr_SetString(PyExc_OverflowError,
24737 eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	7424 "%c arg not in range(0x10000) "
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	7425 "(narrow Python build)");
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	7426 return -1;
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	7427 }
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	7428 #endif
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	7429 buf[0] = (Py_UNICODE) x;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7430 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7431 buf[1] = '\0';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7432 return 1;
12899 9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	7433
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	7434 onError:
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	7435 PyErr_SetString(PyExc_TypeError,
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	7436 "%c requires int or char");
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	7437 return -1;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7438 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7439
13196 18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	7440 /* fmt%(v1,v2,...) is roughly equivalent to sprintf(fmt, v1, v2, ...)
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	7441
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	7442 FORMATBUFLEN is the length of the buffer in which the floats, ints, &
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	7443 chars are formatted. XXX This is a magic number. Each formatting
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	7444 routine does bounds checking to ensure no overflow, but a better
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	7445 solution may be to malloc a buffer of appropriate size for each
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	7446 format. For now, the current solution is sufficient.
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	7447 */
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	7448 #define FORMATBUFLEN (size_t)120
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	7449
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7450 PyObject PyUnicode_Format(PyObject format,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7451 PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7452 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7453 Py_UNICODE fmt, res;
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	7454 Py_ssize_t fmtcnt, rescnt, reslen, arglen, argidx;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7455 int args_owned = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7456 PyUnicodeObject *result = NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7457 PyObject *dict = NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7458 PyObject *uformat;
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	7459
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7460 if (format == NULL \|\| args == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7461 PyErr_BadInternalCall();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7462 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7463 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7464 uformat = PyUnicode_FromObject(format);
12645 c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	7465 if (uformat == NULL)
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	7466 return NULL;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7467 fmt = PyUnicode_AS_UNICODE(uformat);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7468 fmtcnt = PyUnicode_GET_SIZE(uformat);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7469
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7470 reslen = rescnt = fmtcnt + 100;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7471 result = _PyUnicode_New(reslen);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7472 if (result == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7473 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7474 res = PyUnicode_AS_UNICODE(result);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7475
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7476 if (PyTuple_Check(args)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7477 arglen = PyTuple_Size(args);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7478 argidx = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7479 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7480 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7481 arglen = -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7482 argidx = -2;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7483 }
26164 63a08639dae2 Fix SF # 635969, No error "not all arguments converted" Neal Norwitz <nnorwitz@gmail.com> parents: 25895 diff changeset	7484 if (args->ob_type->tp_as_mapping && !PyTuple_Check(args) &&
63a08639dae2 Fix SF # 635969, No error "not all arguments converted" Neal Norwitz <nnorwitz@gmail.com> parents: 25895 diff changeset	7485 !PyObject_TypeCheck(args, &PyBaseString_Type))
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7486 dict = args;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7487
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7488 while (--fmtcnt >= 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7489 if (*fmt != '%') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7490 if (--rescnt < 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7491 rescnt = fmtcnt + 100;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7492 reslen += rescnt;
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	7493 if (_PyUnicode_Resize(&result, reslen) < 0)
36877 e9a7a2784e17 SF #1444030: Fix several potential defects found by Coverity. Hye-Shik Chang <hyeshik@gmail.com> parents: 36675 diff changeset	7494 goto onError;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7495 res = PyUnicode_AS_UNICODE(result) + reslen - rescnt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7496 --rescnt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7497 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7498 res++ = fmt++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7499 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7500 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7501 /* Got a format specifier */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7502 int flags = 0;
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	7503 Py_ssize_t width = -1;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7504 int prec = -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7505 Py_UNICODE c = '\0';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7506 Py_UNICODE fill;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7507 PyObject *v = NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7508 PyObject *temp = NULL;
13196 18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	7509 Py_UNICODE *pbuf;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7510 Py_UNICODE sign;
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	7511 Py_ssize_t len;
13196 18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	7512 Py_UNICODE formatbuf[FORMATBUFLEN]; /* For format{float,int,char}() */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7513
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7514 fmt++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7515 if (*fmt == '(') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7516 Py_UNICODE *keystart;
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	7517 Py_ssize_t keylen;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7518 PyObject *key;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7519 int pcount = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7520
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7521 if (dict == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7522 PyErr_SetString(PyExc_TypeError,
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	7523 "format requires a mapping");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7524 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7525 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7526 ++fmt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7527 --fmtcnt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7528 keystart = fmt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7529 /* Skip over balanced parentheses */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7530 while (pcount > 0 && --fmtcnt >= 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7531 if (*fmt == ')')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7532 --pcount;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7533 else if (*fmt == '(')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7534 ++pcount;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7535 fmt++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7536 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7537 keylen = fmt - keystart - 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7538 if (fmtcnt < 0 \|\| pcount > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7539 PyErr_SetString(PyExc_ValueError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7540 "incomplete format key");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7541 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7542 }
21028 3dfbbf42bf16 Fix for bug #438164: %-formatting using Unicode objects. Marc-André Lemburg <mal@egenix.com> parents: 20563 diff changeset	7543 #if 0
12645 c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	7544 /* keys are converted to strings using UTF-8 and
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7545 then looked up since Python uses strings to hold
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7546 variables names etc. in its namespaces and we
12645 c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	7547 wouldn't want to break common idioms. */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7548 key = PyUnicode_EncodeUTF8(keystart,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7549 keylen,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7550 NULL);
21028 3dfbbf42bf16 Fix for bug #438164: %-formatting using Unicode objects. Marc-André Lemburg <mal@egenix.com> parents: 20563 diff changeset	7551 #else
3dfbbf42bf16 Fix for bug #438164: %-formatting using Unicode objects. Marc-André Lemburg <mal@egenix.com> parents: 20563 diff changeset	7552 key = PyUnicode_FromUnicode(keystart, keylen);
3dfbbf42bf16 Fix for bug #438164: %-formatting using Unicode objects. Marc-André Lemburg <mal@egenix.com> parents: 20563 diff changeset	7553 #endif
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7554 if (key == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7555 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7556 if (args_owned) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7557 Py_DECREF(args);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7558 args_owned = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7559 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7560 args = PyObject_GetItem(dict, key);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7561 Py_DECREF(key);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7562 if (args == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7563 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7564 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7565 args_owned = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7566 arglen = -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7567 argidx = -2;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7568 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7569 while (--fmtcnt >= 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7570 switch (c = *fmt++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7571 case '-': flags \|= F_LJUST; continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7572 case '+': flags \|= F_SIGN; continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7573 case ' ': flags \|= F_BLANK; continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7574 case '#': flags \|= F_ALT; continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7575 case '0': flags \|= F_ZERO; continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7576 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7577 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7578 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7579 if (c == '*') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7580 v = getnextarg(args, arglen, &argidx);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7581 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7582 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7583 if (!PyInt_Check(v)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7584 PyErr_SetString(PyExc_TypeError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7585 "* wants int");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7586 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7587 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7588 width = PyInt_AsLong(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7589 if (width < 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7590 flags \|= F_LJUST;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7591 width = -width;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7592 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7593 if (--fmtcnt >= 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7594 c = *fmt++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7595 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7596 else if (c >= '0' && c <= '9') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7597 width = c - '0';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7598 while (--fmtcnt >= 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7599 c = *fmt++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7600 if (c < '0' \|\| c > '9')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7601 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7602 if ((width*10) / 10 != width) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7603 PyErr_SetString(PyExc_ValueError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7604 "width too big");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7605 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7606 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7607 width = width*10 + (c - '0');
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7608 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7609 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7610 if (c == '.') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7611 prec = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7612 if (--fmtcnt >= 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7613 c = *fmt++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7614 if (c == '*') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7615 v = getnextarg(args, arglen, &argidx);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7616 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7617 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7618 if (!PyInt_Check(v)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7619 PyErr_SetString(PyExc_TypeError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7620 "* wants int");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7621 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7622 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7623 prec = PyInt_AsLong(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7624 if (prec < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7625 prec = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7626 if (--fmtcnt >= 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7627 c = *fmt++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7628 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7629 else if (c >= '0' && c <= '9') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7630 prec = c - '0';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7631 while (--fmtcnt >= 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7632 c = Py_CHARMASK(*fmt++);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7633 if (c < '0' \|\| c > '9')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7634 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7635 if ((prec*10) / 10 != prec) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7636 PyErr_SetString(PyExc_ValueError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7637 "prec too big");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7638 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7639 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7640 prec = prec*10 + (c - '0');
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7641 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7642 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7643 } /* prec */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7644 if (fmtcnt >= 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7645 if (c == 'h' \|\| c == 'l' \|\| c == 'L') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7646 if (--fmtcnt >= 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7647 c = *fmt++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7648 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7649 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7650 if (fmtcnt < 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7651 PyErr_SetString(PyExc_ValueError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7652 "incomplete format");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7653 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7654 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7655 if (c != '%') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7656 v = getnextarg(args, arglen, &argidx);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7657 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7658 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7659 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7660 sign = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7661 fill = ' ';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7662 switch (c) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7663
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7664 case '%':
13196 18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	7665 pbuf = formatbuf;
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	7666 /* presume that buffer length is at least 1 */
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	7667 pbuf[0] = '%';
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7668 len = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7669 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7670
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7671 case 's':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7672 case 'r':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7673 if (PyUnicode_Check(v) && c == 's') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7674 temp = v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7675 Py_INCREF(temp);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7676 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7677 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7678 PyObject *unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7679 if (c == 's')
32801 16de4f90ef44 Let u'%s' % obj try obj.__unicode__() first and fallback to obj.__str__(). Marc-André Lemburg <mal@egenix.com> parents: 32695 diff changeset	7680 temp = PyObject_Unicode(v);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7681 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7682 temp = PyObject_Repr(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7683 if (temp == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7684 goto onError;
32801 16de4f90ef44 Let u'%s' % obj try obj.__unicode__() first and fallback to obj.__str__(). Marc-André Lemburg <mal@egenix.com> parents: 32695 diff changeset	7685 if (PyUnicode_Check(temp))
16de4f90ef44 Let u'%s' % obj try obj.__unicode__() first and fallback to obj.__str__(). Marc-André Lemburg <mal@egenix.com> parents: 32695 diff changeset	7686 /* nothing to do */;
16de4f90ef44 Let u'%s' % obj try obj.__unicode__() first and fallback to obj.__str__(). Marc-André Lemburg <mal@egenix.com> parents: 32695 diff changeset	7687 else if (PyString_Check(temp)) {
16de4f90ef44 Let u'%s' % obj try obj.__unicode__() first and fallback to obj.__str__(). Marc-André Lemburg <mal@egenix.com> parents: 32695 diff changeset	7688 /* convert to string to Unicode */
36945 0f19a6fc9756 - Reindent a confusingly indented piece of code (no intended code changes Thomas Wouters <thomas@python.org> parents: 36924 diff changeset	7689 unicode = PyUnicode_Decode(PyString_AS_STRING(temp),
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7690 PyString_GET_SIZE(temp),
36945 0f19a6fc9756 - Reindent a confusingly indented piece of code (no intended code changes Thomas Wouters <thomas@python.org> parents: 36924 diff changeset	7691 NULL,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7692 "strict");
36945 0f19a6fc9756 - Reindent a confusingly indented piece of code (no intended code changes Thomas Wouters <thomas@python.org> parents: 36924 diff changeset	7693 Py_DECREF(temp);
0f19a6fc9756 - Reindent a confusingly indented piece of code (no intended code changes Thomas Wouters <thomas@python.org> parents: 36924 diff changeset	7694 temp = unicode;
0f19a6fc9756 - Reindent a confusingly indented piece of code (no intended code changes Thomas Wouters <thomas@python.org> parents: 36924 diff changeset	7695 if (temp == NULL)
0f19a6fc9756 - Reindent a confusingly indented piece of code (no intended code changes Thomas Wouters <thomas@python.org> parents: 36924 diff changeset	7696 goto onError;
0f19a6fc9756 - Reindent a confusingly indented piece of code (no intended code changes Thomas Wouters <thomas@python.org> parents: 36924 diff changeset	7697 }
32801 16de4f90ef44 Let u'%s' % obj try obj.__unicode__() first and fallback to obj.__str__(). Marc-André Lemburg <mal@egenix.com> parents: 32695 diff changeset	7698 else {
16de4f90ef44 Let u'%s' % obj try obj.__unicode__() first and fallback to obj.__str__(). Marc-André Lemburg <mal@egenix.com> parents: 32695 diff changeset	7699 Py_DECREF(temp);
16de4f90ef44 Let u'%s' % obj try obj.__unicode__() first and fallback to obj.__str__(). Marc-André Lemburg <mal@egenix.com> parents: 32695 diff changeset	7700 PyErr_SetString(PyExc_TypeError,
16de4f90ef44 Let u'%s' % obj try obj.__unicode__() first and fallback to obj.__str__(). Marc-André Lemburg <mal@egenix.com> parents: 32695 diff changeset	7701 "%s argument has non-string str()");
16de4f90ef44 Let u'%s' % obj try obj.__unicode__() first and fallback to obj.__str__(). Marc-André Lemburg <mal@egenix.com> parents: 32695 diff changeset	7702 goto onError;
16de4f90ef44 Let u'%s' % obj try obj.__unicode__() first and fallback to obj.__str__(). Marc-André Lemburg <mal@egenix.com> parents: 32695 diff changeset	7703 }
16de4f90ef44 Let u'%s' % obj try obj.__unicode__() first and fallback to obj.__str__(). Marc-André Lemburg <mal@egenix.com> parents: 32695 diff changeset	7704 }
13196 18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	7705 pbuf = PyUnicode_AS_UNICODE(temp);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7706 len = PyUnicode_GET_SIZE(temp);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7707 if (prec >= 0 && len > prec)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7708 len = prec;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7709 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7710
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7711 case 'i':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7712 case 'd':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7713 case 'u':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7714 case 'o':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7715 case 'x':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7716 case 'X':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7717 if (c == 'i')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7718 c = 'd';
15455 d0146d128897 Fox for SF bug #123859: %[duxXo] long formats inconsistent. Tim Peters <tim.peters@gmail.com> parents: 15001 diff changeset	7719 if (PyLong_Check(v)) {
14713 58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	7720 temp = formatlong(v, flags, prec, c);
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	7721 if (!temp)
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	7722 goto onError;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	7723 pbuf = PyUnicode_AS_UNICODE(temp);
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	7724 len = PyUnicode_GET_SIZE(temp);
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	7725 sign = 1;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	7726 }
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	7727 else {
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	7728 pbuf = formatbuf;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	7729 len = formatint(pbuf, sizeof(formatbuf)/sizeof(Py_UNICODE),
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	7730 flags, prec, c, v);
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	7731 if (len < 0)
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	7732 goto onError;
31042 9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions Guido van Rossum <guido@python.org> parents: 31021 diff changeset	7733 sign = 1;
14713 58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	7734 }
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	7735 if (flags & F_ZERO)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7736 fill = '0';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7737 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7738
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7739 case 'e':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7740 case 'E':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7741 case 'f':
30216 7ab1a41f16da SF bug #795506: Wrong handling of string format code for float values. Raymond Hettinger <python@rcn.com> parents: 30164 diff changeset	7742 case 'F':
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7743 case 'g':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7744 case 'G':
30216 7ab1a41f16da SF bug #795506: Wrong handling of string format code for float values. Raymond Hettinger <python@rcn.com> parents: 30164 diff changeset	7745 if (c == 'F')
7ab1a41f16da SF bug #795506: Wrong handling of string format code for float values. Raymond Hettinger <python@rcn.com> parents: 30164 diff changeset	7746 c = 'f';
13196 18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	7747 pbuf = formatbuf;
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	7748 len = formatfloat(pbuf, sizeof(formatbuf)/sizeof(Py_UNICODE),
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	7749 flags, prec, c, v);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7750 if (len < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7751 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7752 sign = 1;
14713 58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	7753 if (flags & F_ZERO)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7754 fill = '0';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7755 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7756
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7757 case 'c':
13196 18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	7758 pbuf = formatbuf;
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	7759 len = formatchar(pbuf, sizeof(formatbuf)/sizeof(Py_UNICODE), v);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7760 if (len < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7761 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7762 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7763
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7764 default:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7765 PyErr_Format(PyExc_ValueError,
15579 31d78f361530 [ Patch #102852 ] Make % error a bit more informative by indicates the Andrew M. Kuchling <amk@amk.ca> parents: 15455 diff changeset	7766 "unsupported format character '%c' (0x%x) "
39995 6ba402b60041 Forward-port of r52136,52138: a review of overflow-detecting code. Armin Rigo <arigo@tunes.org> parents: 39734 diff changeset	7767 "at index %zd",
30441 76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type, Tim Peters <tim.peters@gmail.com> parents: 30439 diff changeset	7768 (31<=c && c<=126) ? (char)c : '?',
25375 4df0e691f4da Add cast to avoid compiler warning. Marc-André Lemburg <mal@egenix.com> parents: 25248 diff changeset	7769 (int)c,
39995 6ba402b60041 Forward-port of r52136,52138: a review of overflow-detecting code. Armin Rigo <arigo@tunes.org> parents: 39734 diff changeset	7770 (Py_ssize_t)(fmt - 1 -
6ba402b60041 Forward-port of r52136,52138: a review of overflow-detecting code. Armin Rigo <arigo@tunes.org> parents: 39734 diff changeset	7771 PyUnicode_AS_UNICODE(uformat)));
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7772 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7773 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7774 if (sign) {
13196 18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	7775 if (pbuf == '-' \|\| pbuf == '+') {
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	7776 sign = *pbuf++;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7777 len--;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7778 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7779 else if (flags & F_SIGN)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7780 sign = '+';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7781 else if (flags & F_BLANK)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7782 sign = ' ';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7783 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7784 sign = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7785 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7786 if (width < len)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7787 width = len;
25719 f532e9b69551 Fix a nasty endcase reported by Armin Rigo in SF bug 618623: Guido van Rossum <guido@python.org> parents: 25375 diff changeset	7788 if (rescnt - (sign != 0) < width) {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7789 reslen -= rescnt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7790 rescnt = width + fmtcnt + 100;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7791 reslen += rescnt;
25719 f532e9b69551 Fix a nasty endcase reported by Armin Rigo in SF bug 618623: Guido van Rossum <guido@python.org> parents: 25375 diff changeset	7792 if (reslen < 0) {
36877 e9a7a2784e17 SF #1444030: Fix several potential defects found by Coverity. Hye-Shik Chang <hyeshik@gmail.com> parents: 36675 diff changeset	7793 Py_XDECREF(temp);
36945 0f19a6fc9756 - Reindent a confusingly indented piece of code (no intended code changes Thomas Wouters <thomas@python.org> parents: 36924 diff changeset	7794 PyErr_NoMemory();
0f19a6fc9756 - Reindent a confusingly indented piece of code (no intended code changes Thomas Wouters <thomas@python.org> parents: 36924 diff changeset	7795 goto onError;
25719 f532e9b69551 Fix a nasty endcase reported by Armin Rigo in SF bug 618623: Guido van Rossum <guido@python.org> parents: 25375 diff changeset	7796 }
36945 0f19a6fc9756 - Reindent a confusingly indented piece of code (no intended code changes Thomas Wouters <thomas@python.org> parents: 36924 diff changeset	7797 if (_PyUnicode_Resize(&result, reslen) < 0) {
0f19a6fc9756 - Reindent a confusingly indented piece of code (no intended code changes Thomas Wouters <thomas@python.org> parents: 36924 diff changeset	7798 Py_XDECREF(temp);
0f19a6fc9756 - Reindent a confusingly indented piece of code (no intended code changes Thomas Wouters <thomas@python.org> parents: 36924 diff changeset	7799 goto onError;
0f19a6fc9756 - Reindent a confusingly indented piece of code (no intended code changes Thomas Wouters <thomas@python.org> parents: 36924 diff changeset	7800 }
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7801 res = PyUnicode_AS_UNICODE(result)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7802 + reslen - rescnt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7803 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7804 if (sign) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7805 if (fill != ' ')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7806 *res++ = sign;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7807 rescnt--;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7808 if (width > len)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7809 width--;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7810 }
14713 58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	7811 if ((flags & F_ALT) && (c == 'x' \|\| c == 'X')) {
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	7812 assert(pbuf[0] == '0');
17450 f74f5aafdc7e Bug 415514 reported that e.g. Tim Peters <tim.peters@gmail.com> parents: 17425 diff changeset	7813 assert(pbuf[1] == c);
f74f5aafdc7e Bug 415514 reported that e.g. Tim Peters <tim.peters@gmail.com> parents: 17425 diff changeset	7814 if (fill != ' ') {
f74f5aafdc7e Bug 415514 reported that e.g. Tim Peters <tim.peters@gmail.com> parents: 17425 diff changeset	7815 res++ = pbuf++;
f74f5aafdc7e Bug 415514 reported that e.g. Tim Peters <tim.peters@gmail.com> parents: 17425 diff changeset	7816 res++ = pbuf++;
14713 58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	7817 }
17450 f74f5aafdc7e Bug 415514 reported that e.g. Tim Peters <tim.peters@gmail.com> parents: 17425 diff changeset	7818 rescnt -= 2;
f74f5aafdc7e Bug 415514 reported that e.g. Tim Peters <tim.peters@gmail.com> parents: 17425 diff changeset	7819 width -= 2;
f74f5aafdc7e Bug 415514 reported that e.g. Tim Peters <tim.peters@gmail.com> parents: 17425 diff changeset	7820 if (width < 0)
f74f5aafdc7e Bug 415514 reported that e.g. Tim Peters <tim.peters@gmail.com> parents: 17425 diff changeset	7821 width = 0;
f74f5aafdc7e Bug 415514 reported that e.g. Tim Peters <tim.peters@gmail.com> parents: 17425 diff changeset	7822 len -= 2;
14713 58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	7823 }
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7824 if (width > len && !(flags & F_LJUST)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7825 do {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7826 --rescnt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7827 *res++ = fill;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7828 } while (--width > len);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7829 }
14713 58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	7830 if (fill == ' ') {
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	7831 if (sign)
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	7832 *res++ = sign;
17450 f74f5aafdc7e Bug 415514 reported that e.g. Tim Peters <tim.peters@gmail.com> parents: 17425 diff changeset	7833 if ((flags & F_ALT) && (c == 'x' \|\| c == 'X')) {
14713 58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	7834 assert(pbuf[0] == '0');
17450 f74f5aafdc7e Bug 415514 reported that e.g. Tim Peters <tim.peters@gmail.com> parents: 17425 diff changeset	7835 assert(pbuf[1] == c);
14713 58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	7836 res++ = pbuf++;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	7837 res++ = pbuf++;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	7838 }
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	7839 }
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	7840 Py_UNICODE_COPY(res, pbuf, len);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7841 res += len;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7842 rescnt -= len;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7843 while (--width >= len) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7844 --rescnt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7845 *res++ = ' ';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7846 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7847 if (dict && (argidx < arglen) && c != '%') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7848 PyErr_SetString(PyExc_TypeError,
23494 76f376bea5e5 Patch 549187. Improve string formatting error message. Raymond Hettinger <python@rcn.com> parents: 23295 diff changeset	7849 "not all arguments converted during string formatting");
36945 0f19a6fc9756 - Reindent a confusingly indented piece of code (no intended code changes Thomas Wouters <thomas@python.org> parents: 36924 diff changeset	7850 Py_XDECREF(temp);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7851 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7852 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7853 Py_XDECREF(temp);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7854 } /* '%' */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7855 } /* until end */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7856 if (argidx < arglen && !dict) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7857 PyErr_SetString(PyExc_TypeError,
23494 76f376bea5e5 Patch 549187. Improve string formatting error message. Raymond Hettinger <python@rcn.com> parents: 23295 diff changeset	7858 "not all arguments converted during string formatting");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7859 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7860 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7861
36945 0f19a6fc9756 - Reindent a confusingly indented piece of code (no intended code changes Thomas Wouters <thomas@python.org> parents: 36924 diff changeset	7862 if (_PyUnicode_Resize(&result, reslen - rescnt) < 0)
0f19a6fc9756 - Reindent a confusingly indented piece of code (no intended code changes Thomas Wouters <thomas@python.org> parents: 36924 diff changeset	7863 goto onError;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7864 if (args_owned) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7865 Py_DECREF(args);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7866 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7867 Py_DECREF(uformat);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7868 return (PyObject *)result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7869
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7870 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7871 Py_XDECREF(result);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7872 Py_DECREF(uformat);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7873 if (args_owned) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7874 Py_DECREF(args);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7875 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7876 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7877 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7878
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7879 static PyBufferProcs unicode_as_buffer = {
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	7880 (readbufferproc) unicode_buffer_getreadbuf,
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	7881 (writebufferproc) unicode_buffer_getwritebuf,
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	7882 (segcountproc) unicode_buffer_getsegcount,
7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	7883 (charbufferproc) unicode_buffer_getcharbuf,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7884 };
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7885
24280 1598a1e473e2 staticforward bites the dust. Jeremy Hylton <jeremy@alum.mit.edu> parents: 24071 diff changeset	7886 static PyObject *
19593 a80b4b006eb7 Make unicode subclassable. Guido van Rossum <guido@python.org> parents: 19301 diff changeset	7887 unicode_subtype_new(PyTypeObject type, PyObject args, PyObject *kwds);
a80b4b006eb7 Make unicode subclassable. Guido van Rossum <guido@python.org> parents: 19301 diff changeset	7888
18933 09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	7889 static PyObject *
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	7890 unicode_new(PyTypeObject type, PyObject args, PyObject *kwds)
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	7891 {
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	7892 PyObject *x = NULL;
36675 dba6494735d0 Revert backwards-incompatible const changes. Martin v. Löwis <martin@v.loewis.de> parents: 36532 diff changeset	7893 static char *kwlist[] = {"string", "encoding", "errors", 0};
18933 09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	7894 char *encoding = NULL;
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	7895 char *errors = NULL;
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	7896
19593 a80b4b006eb7 Make unicode subclassable. Guido van Rossum <guido@python.org> parents: 19301 diff changeset	7897 if (type != &PyUnicode_Type)
a80b4b006eb7 Make unicode subclassable. Guido van Rossum <guido@python.org> parents: 19301 diff changeset	7898 return unicode_subtype_new(type, args, kwds);
18933 09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	7899 if (!PyArg_ParseTupleAndKeywords(args, kwds, "\|Oss:unicode",
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	7900 kwlist, &x, &encoding, &errors))
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	7901 return NULL;
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	7902 if (x == NULL)
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	7903 return (PyObject *)_PyUnicode_New(0);
20554 0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	7904 if (encoding == NULL && errors == NULL)
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	7905 return PyObject_Unicode(x);
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	7906 else
18933 09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	7907 return PyUnicode_FromEncodedObject(x, encoding, errors);
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	7908 }
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	7909
19593 a80b4b006eb7 Make unicode subclassable. Guido van Rossum <guido@python.org> parents: 19301 diff changeset	7910 static PyObject *
a80b4b006eb7 Make unicode subclassable. Guido van Rossum <guido@python.org> parents: 19301 diff changeset	7911 unicode_subtype_new(PyTypeObject type, PyObject args, PyObject *kwds)
a80b4b006eb7 Make unicode subclassable. Guido van Rossum <guido@python.org> parents: 19301 diff changeset	7912 {
19886 3b0da13ace48 str_subtype_new, unicode_subtype_new: Tim Peters <tim.peters@gmail.com> parents: 19885 diff changeset	7913 PyUnicodeObject tmp, pnew;
36501 7fdc639bc5b4 Merge ssize_t branch. Martin v. Löwis <martin@v.loewis.de> parents: 36261 diff changeset	7914 Py_ssize_t n;
19593 a80b4b006eb7 Make unicode subclassable. Guido van Rossum <guido@python.org> parents: 19301 diff changeset	7915
a80b4b006eb7 Make unicode subclassable. Guido van Rossum <guido@python.org> parents: 19301 diff changeset	7916 assert(PyType_IsSubtype(type, &PyUnicode_Type));
a80b4b006eb7 Make unicode subclassable. Guido van Rossum <guido@python.org> parents: 19301 diff changeset	7917 tmp = (PyUnicodeObject *)unicode_new(&PyUnicode_Type, args, kwds);
a80b4b006eb7 Make unicode subclassable. Guido van Rossum <guido@python.org> parents: 19301 diff changeset	7918 if (tmp == NULL)
a80b4b006eb7 Make unicode subclassable. Guido van Rossum <guido@python.org> parents: 19301 diff changeset	7919 return NULL;
a80b4b006eb7 Make unicode subclassable. Guido van Rossum <guido@python.org> parents: 19301 diff changeset	7920 assert(PyUnicode_Check(tmp));
19886 3b0da13ace48 str_subtype_new, unicode_subtype_new: Tim Peters <tim.peters@gmail.com> parents: 19885 diff changeset	7921 pnew = (PyUnicodeObject *) type->tp_alloc(type, n = tmp->length);
29641 4909aa64405f SF patch 703666: Several objects don't decref tmp on failure in subtype_new Raymond Hettinger <python@rcn.com> parents: 29164 diff changeset	7922 if (pnew == NULL) {
4909aa64405f SF patch 703666: Several objects don't decref tmp on failure in subtype_new Raymond Hettinger <python@rcn.com> parents: 29164 diff changeset	7923 Py_DECREF(tmp);
19593 a80b4b006eb7 Make unicode subclassable. Guido van Rossum <guido@python.org> parents: 19301 diff changeset	7924 return NULL;
29641 4909aa64405f SF patch 703666: Several objects don't decref tmp on failure in subtype_new Raymond Hettinger <python@rcn.com> parents: 29164 diff changeset	7925 }
19886 3b0da13ace48 str_subtype_new, unicode_subtype_new: Tim Peters <tim.peters@gmail.com> parents: 19885 diff changeset	7926 pnew->str = PyMem_NEW(Py_UNICODE, n+1);
3b0da13ace48 str_subtype_new, unicode_subtype_new: Tim Peters <tim.peters@gmail.com> parents: 19885 diff changeset	7927 if (pnew->str == NULL) {
3b0da13ace48 str_subtype_new, unicode_subtype_new: Tim Peters <tim.peters@gmail.com> parents: 19885 diff changeset	7928 _Py_ForgetReference((PyObject *)pnew);
22981 dc1fb19d990d Remove PyMalloc_. Neil Schemenauer <nascheme@enme.ucalgary.ca>* parents: 22938 diff changeset	7929 PyObject_Del(pnew);
29641 4909aa64405f SF patch 703666: Several objects don't decref tmp on failure in subtype_new Raymond Hettinger <python@rcn.com> parents: 29164 diff changeset	7930 Py_DECREF(tmp);
27826 7dffcc498c45 Add more missing PyErr_NoMemory() after failled memory allocs Neal Norwitz <nnorwitz@gmail.com> parents: 27774 diff changeset	7931 return PyErr_NoMemory();
19593 a80b4b006eb7 Make unicode subclassable. Guido van Rossum <guido@python.org> parents: 19301 diff changeset	7932 }
19886 3b0da13ace48 str_subtype_new, unicode_subtype_new: Tim Peters <tim.peters@gmail.com> parents: 19885 diff changeset	7933 Py_UNICODE_COPY(pnew->str, tmp->str, n+1);
3b0da13ace48 str_subtype_new, unicode_subtype_new: Tim Peters <tim.peters@gmail.com> parents: 19885 diff changeset	7934 pnew->length = n;
3b0da13ace48 str_subtype_new, unicode_subtype_new: Tim Peters <tim.peters@gmail.com> parents: 19885 diff changeset	7935 pnew->hash = tmp->hash;
19593 a80b4b006eb7 Make unicode subclassable. Guido van Rossum <guido@python.org> parents: 19301 diff changeset	7936 Py_DECREF(tmp);
19886 3b0da13ace48 str_subtype_new, unicode_subtype_new: Tim Peters <tim.peters@gmail.com> parents: 19885 diff changeset	7937 return (PyObject *)pnew;
19593 a80b4b006eb7 Make unicode subclassable. Guido van Rossum <guido@python.org> parents: 19301 diff changeset	7938 }
a80b4b006eb7 Make unicode subclassable. Guido van Rossum <guido@python.org> parents: 19301 diff changeset	7939
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	7940 PyDoc_STRVAR(unicode_doc,
18933 09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	7941 "unicode(string [, encoding[, errors]]) -> object\n\
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	7942 \n\
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	7943 Create a new Unicode object from the given encoded string.\n\
24392 0a327b768868 tighten up the unicode object's docstring a tad Skip Montanaro <skip@pobox.com> parents: 24280 diff changeset	7944 encoding defaults to the current default string encoding.\n\
0a327b768868 tighten up the unicode object's docstring a tad Skip Montanaro <skip@pobox.com> parents: 24280 diff changeset	7945 errors can be 'strict', 'replace' or 'ignore' and defaults to 'strict'.");
18933 09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	7946
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7947 PyTypeObject PyUnicode_Type = {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7948 PyObject_HEAD_INIT(&PyType_Type)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7949 0, /* ob_size */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7950 "unicode", /* tp_name */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7951 sizeof(PyUnicodeObject), /* tp_size */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7952 0, /* tp_itemsize */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7953 /* Slots */
20277 2983b54256c7 Enable GC for new-style instances. This touches lots of files, since Guido van Rossum <guido@python.org> parents: 20018 diff changeset	7954 (destructor)unicode_dealloc, /* tp_dealloc */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7955 0, /* tp_print */
18933 09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	7956 0, /* tp_getattr */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7957 0, /* tp_setattr */
39487 635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	7958 0, /* tp_compare */
37304 79a0f080f898 Remove unnecessary casts in type object initializers. Georg Brandl <georg@python.org> parents: 36945 diff changeset	7959 unicode_repr, /* tp_repr */
26274 958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 26164 diff changeset	7960 &unicode_as_number, /* tp_as_number */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7961 &unicode_as_sequence, /* tp_as_sequence */
23778 245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	7962 &unicode_as_mapping, /* tp_as_mapping */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7963 (hashfunc) unicode_hash, /* tp_hash*/
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7964 0, /* tp_call*/
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7965 (reprfunc) unicode_str, /* tp_str */
18933 09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	7966 PyObject_GenericGetAttr, /* tp_getattro */
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	7967 0, /* tp_setattro */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7968 &unicode_as_buffer, /* tp_as_buffer */
26274 958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 26164 diff changeset	7969 Py_TPFLAGS_DEFAULT \| Py_TPFLAGS_CHECKTYPES \|
40908 d01bd37bf5ce Variation of patch # 1624059 to speed up checking if an object is a subclass Neal Norwitz <nnorwitz@gmail.com> parents: 39995 diff changeset	7970 Py_TPFLAGS_BASETYPE \| Py_TPFLAGS_UNICODE_SUBCLASS, /* tp_flags */
18933 09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	7971 unicode_doc, /* tp_doc */
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	7972 0, /* tp_traverse */
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	7973 0, /* tp_clear */
39487 635b09b1f542 Slightly revised version of patch #1538956: Marc-André Lemburg <mal@egenix.com> parents: 39463 diff changeset	7974 PyUnicode_RichCompare, /* tp_richcompare */
18933 09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	7975 0, /* tp_weaklistoffset */
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	7976 0, /* tp_iter */
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	7977 0, /* tp_iternext */
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	7978 unicode_methods, /* tp_methods */
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	7979 0, /* tp_members */
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	7980 0, /* tp_getset */
23542 220b98f06415 - A new type object, 'string', is added. This is a common base type Guido van Rossum <guido@python.org> parents: 23494 diff changeset	7981 &PyBaseString_Type, /* tp_base */
18933 09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	7982 0, /* tp_dict */
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	7983 0, /* tp_descr_get */
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	7984 0, /* tp_descr_set */
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	7985 0, /* tp_dictoffset */
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	7986 0, /* tp_init */
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	7987 0, /* tp_alloc */
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	7988 unicode_new, /* tp_new */
22981 dc1fb19d990d Remove PyMalloc_. Neil Schemenauer <nascheme@enme.ucalgary.ca>* parents: 22938 diff changeset	7989 PyObject_Del, /* tp_free */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7990 };
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7991
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7992 /* Initialize the Unicode implementation */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7993
13660 a1ce07218781 Miscelaneous ANSIfications. I'm assuming here 'main' should take (int, Thomas Wouters <thomas@python.org> parents: 13635 diff changeset	7994 void _PyUnicode_Init(void)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	7995 {
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	7996 int i;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	7997
38257 af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	7998 /* XXX - move this array to unicodectype.c ? */
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	7999 Py_UNICODE linebreak[] = {
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	8000 0x000A, /* LINE FEED */
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	8001 0x000D, /* CARRIAGE RETURN */
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	8002 0x001C, /* FILE SEPARATOR */
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	8003 0x001D, /* GROUP SEPARATOR */
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	8004 0x001E, /* RECORD SEPARATOR */
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	8005 0x0085, /* NEXT LINE */
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	8006 0x2028, /* LINE SEPARATOR */
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	8007 0x2029, /* PARAGRAPH SEPARATOR */
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	8008 };
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	8009
12645 c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	8010 /* Init the implementation */
12899 9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	8011 unicode_freelist = NULL;
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	8012 unicode_freelist_size = 0;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	8013 unicode_empty = _PyUnicode_New(0);
39129 f2031f15d99d Handle allocation failures gracefully. Found with failmalloc. Neal Norwitz <nnorwitz@gmail.com> parents: 38817 diff changeset	8014 if (!unicode_empty)
f2031f15d99d Handle allocation failures gracefully. Found with failmalloc. Neal Norwitz <nnorwitz@gmail.com> parents: 38817 diff changeset	8015 return;
f2031f15d99d Handle allocation failures gracefully. Found with failmalloc. Neal Norwitz <nnorwitz@gmail.com> parents: 38817 diff changeset	8016
12876 243593f11fcf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12646 diff changeset	8017 strcpy(unicode_default_encoding, "ascii");
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	8018 for (i = 0; i < 256; i++)
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	8019 unicode_latin1[i] = NULL;
23542 220b98f06415 - A new type object, 'string', is added. This is a common base type Guido van Rossum <guido@python.org> parents: 23494 diff changeset	8020 if (PyType_Ready(&PyUnicode_Type) < 0)
220b98f06415 - A new type object, 'string', is added. This is a common base type Guido van Rossum <guido@python.org> parents: 23494 diff changeset	8021 Py_FatalError("Can't initialize 'unicode'");
38257 af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	8022
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	8023 /* initialize the linebreak bloom filter */
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	8024 bloom_linebreak = make_bloom_mask(
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	8025 linebreak, sizeof(linebreak) / sizeof(linebreak[0])
af46c7ed05eb needforspeed: use append+reverse for rsplit, use "bloom filters" to Fredrik Lundh <fredrik@pythonware.com> parents: 38243 diff changeset	8026 );
38786 9c52efcaff65 Initialize the type object so pychecker can't crash the interpreter. Neal Norwitz <nnorwitz@gmail.com> parents: 38709 diff changeset	8027
9c52efcaff65 Initialize the type object so pychecker can't crash the interpreter. Neal Norwitz <nnorwitz@gmail.com> parents: 38709 diff changeset	8028 PyType_Ready(&EncodingMapType);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	8029 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	8030
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	8031 /* Finalize the Unicode implementation */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	8032
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	8033 void
13660 a1ce07218781 Miscelaneous ANSIfications. I'm assuming here 'main' should take (int, Thomas Wouters <thomas@python.org> parents: 13635 diff changeset	8034 _PyUnicode_Fini(void)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	8035 {
15001 a349b89bf10e _PyUnicode_Fini(): Initialize the local freelist walking variable `u' Barry Warsaw <barry@python.org> parents: 14998 diff changeset	8036 PyUnicodeObject *u;
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	8037 int i;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	8038
14998 c16bdf555c79 In _PyUnicode_Fini(), decref unicode_empty before tearng down the free Guido van Rossum <guido@python.org> parents: 14869 diff changeset	8039 Py_XDECREF(unicode_empty);
c16bdf555c79 In _PyUnicode_Fini(), decref unicode_empty before tearng down the free Guido van Rossum <guido@python.org> parents: 14869 diff changeset	8040 unicode_empty = NULL;
15001 a349b89bf10e _PyUnicode_Fini(): Initialize the local freelist walking variable `u' Barry Warsaw <barry@python.org> parents: 14998 diff changeset	8041
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	8042 for (i = 0; i < 256; i++) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	8043 if (unicode_latin1[i]) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	8044 Py_DECREF(unicode_latin1[i]);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	8045 unicode_latin1[i] = NULL;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	8046 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	8047 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	8048
15001 a349b89bf10e _PyUnicode_Fini(): Initialize the local freelist walking variable `u' Barry Warsaw <barry@python.org> parents: 14998 diff changeset	8049 for (u = unicode_freelist; u != NULL;) {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	8050 PyUnicodeObject *v = u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	8051 u = (PyUnicodeObject *)u;
12329 3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	8052 if (v->str)
12576 1c7c2dd1beb1 Vladimir Marangozov's long-awaited malloc restructuring. Guido van Rossum <guido@python.org> parents: 12568 diff changeset	8053 PyMem_DEL(v->str);
13837 2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	8054 Py_XDECREF(v->defenc);
22981 dc1fb19d990d Remove PyMalloc_. Neil Schemenauer <nascheme@enme.ucalgary.ca>* parents: 22938 diff changeset	8055 PyObject_Del(v);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	8056 }
12899 9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	8057 unicode_freelist = NULL;
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	8058 unicode_freelist_size = 0;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	8059 }
29164 4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	8060
37655 2f3f2e32dfd9 spread the extern "C" { } magic pixie dust around. Python itself builds now Anthony Baxter <anthonybaxter@gmail.com> parents: 37596 diff changeset	8061 #ifdef __cplusplus
2f3f2e32dfd9 spread the extern "C" { } magic pixie dust around. Python itself builds now Anthony Baxter <anthonybaxter@gmail.com> parents: 37596 diff changeset	8062 }
2f3f2e32dfd9 spread the extern "C" { } magic pixie dust around. Python itself builds now Anthony Baxter <anthonybaxter@gmail.com> parents: 37596 diff changeset	8063 #endif
2f3f2e32dfd9 spread the extern "C" { } magic pixie dust around. Python itself builds now Anthony Baxter <anthonybaxter@gmail.com> parents: 37596 diff changeset	8064
2f3f2e32dfd9 spread the extern "C" { } magic pixie dust around. Python itself builds now Anthony Baxter <anthonybaxter@gmail.com> parents: 37596 diff changeset	8065
29164 4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	8066 /*
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	8067 Local variables:
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	8068 c-basic-offset: 4
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	8069 indent-tabs-mode: nil
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	8070 End:
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	8071 */

Mercurial > cpython-lru_cache-weakref

annotate Objects/unicodeobject.c @ 41794:2c00a1910271 legacy-trunk