Mercurial > cpython-lru_cache-weakref

annotate Modules/_codecsmodule.c @ 73505:bcd347cd6bf2

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Use unittest.skipUnless to skip the test related to the glibc bug, issue #13309.
author Florent Xicluna <florent.xicluna@gmail.com>
date Fri, 11 Nov 2011 18:59:30 +0100
parents 6e660d9fd509
children 9688c634f0d9
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
1 /* ------------------------------------------------------------------------
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
2
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
3 _codecs -- Provides access to the codec registry and the builtin
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
4 codecs.
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
5
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
6 This module should never be imported directly. The standard library
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
7 module "codecs" wraps this builtin module for use within Python.
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
8
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
9 The codec registry is accessible via:
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
10
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
11 register(search_function) -> None
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
12
42324
4bbbe0c55fa2 Merged revisions 56443-56466 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42124
diff changeset
13 lookup(encoding) -> CodecInfo object
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
14
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
15 The builtin Unicode codecs use the following interface:
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
16
35825
cc17a582518a Whitespace cleanup.
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
17 <encoding>_encode(Unicode_object[,errors='strict']) ->
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
18 (string object, bytes consumed)
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
19
35825
cc17a582518a Whitespace cleanup.
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
20 <encoding>_decode(char_buffer_obj[,errors='strict']) ->
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
21 (Unicode object, bytes consumed)
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
22
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
23 <encoding>_encode() interfaces also accept non-Unicode object as
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
24 input. The objects are then converted to Unicode using
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
25 PyUnicode_FromObject() prior to applying the conversion.
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
26
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
27 These <encoding>s are available: utf_8, unicode_escape,
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
28 raw_unicode_escape, unicode_internal, latin_1, ascii (7-bit),
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
29 mbcs (on win32).
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
30
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
31
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
32 Written by Marc-Andre Lemburg (mal@lemburg.com).
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
33
13828
254f8b7c4a83 Changing the CNRI copyright notice according to CNRI's instructions.
Guido van Rossum <guido@python.org>
parents: 13651
diff changeset
34 Copyright (c) Corporation for National Research Initiatives.
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
35
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
36 ------------------------------------------------------------------------ */
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
37
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 35825
diff changeset
38 #define PY_SSIZE_T_CLEAN
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
39 #include "Python.h"
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
40
73479
27f5757cca0b Fix build on Windows
Victor Stinner <victor.stinner@haypocalc.com>
parents: 73476
diff changeset
41 #ifdef MS_WINDOWS
27f5757cca0b Fix build on Windows
Victor Stinner <victor.stinner@haypocalc.com>
parents: 73476
diff changeset
42 #include <windows.h>
27f5757cca0b Fix build on Windows
Victor Stinner <victor.stinner@haypocalc.com>
parents: 73476
diff changeset
43 #endif
27f5757cca0b Fix build on Windows
Victor Stinner <victor.stinner@haypocalc.com>
parents: 73476
diff changeset
44
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
45 /* --- Registry ----------------------------------------------------------- */
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
46
25974
040af9c736bb Add docstrings to register, lookup, register_error
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
47 PyDoc_STRVAR(register__doc__,
040af9c736bb Add docstrings to register, lookup, register_error
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
48 "register(search_function)\n\
040af9c736bb Add docstrings to register, lookup, register_error
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
49 \n\
040af9c736bb Add docstrings to register, lookup, register_error
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
50 Register a codec search function. Search functions are expected to take\n\
040af9c736bb Add docstrings to register, lookup, register_error
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
51 one argument, the encoding name in all lower case letters, and return\n\
42324
4bbbe0c55fa2 Merged revisions 56443-56466 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42124
diff changeset
52 a tuple of functions (encoder, decoder, stream_reader, stream_writer)\n\
4bbbe0c55fa2 Merged revisions 56443-56466 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42124
diff changeset
53 (or a CodecInfo object).");
25974
040af9c736bb Add docstrings to register, lookup, register_error
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
54
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
55 static
38669
19fd6141940d Partially merge trunk into p3yk. The removal of Mac/Tools is confusing svn
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
56 PyObject *codec_register(PyObject *self, PyObject *search_function)
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
57 {
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
58 if (PyCodec_Register(search_function))
38669
19fd6141940d Partially merge trunk into p3yk. The removal of Mac/Tools is confusing svn
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
59 return NULL;
35825
cc17a582518a Whitespace cleanup.
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
60
38669
19fd6141940d Partially merge trunk into p3yk. The removal of Mac/Tools is confusing svn
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
61 Py_RETURN_NONE;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
62 }
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
63
25974
040af9c736bb Add docstrings to register, lookup, register_error
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
64 PyDoc_STRVAR(lookup__doc__,
42324
4bbbe0c55fa2 Merged revisions 56443-56466 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42124
diff changeset
65 "lookup(encoding) -> CodecInfo\n\
25974
040af9c736bb Add docstrings to register, lookup, register_error
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
66 \n\
040af9c736bb Add docstrings to register, lookup, register_error
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
67 Looks up a codec tuple in the Python codec registry and returns\n\
52260
c6688e9156b5 Merged revisions 70507-70508 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 50975
diff changeset
68 a CodecInfo object.");
25974
040af9c736bb Add docstrings to register, lookup, register_error
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
69
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
70 static
32572
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
71 PyObject *codec_lookup(PyObject *self, PyObject *args)
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
72 {
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
73 char *encoding;
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
74
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
75 if (!PyArg_ParseTuple(args, "s:lookup", &encoding))
38669
19fd6141940d Partially merge trunk into p3yk. The removal of Mac/Tools is confusing svn
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
76 return NULL;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
77
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
78 return _PyCodec_Lookup(encoding);
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
79 }
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
80
32572
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
81 PyDoc_STRVAR(encode__doc__,
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
82 "encode(obj, [encoding[,errors]]) -> object\n\
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
83 \n\
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
84 Encodes obj using the codec registered for encoding. encoding defaults\n\
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
85 to the default encoding. errors may be given to set a different error\n\
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
86 handling scheme. Default is 'strict' meaning that encoding errors raise\n\
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
87 a ValueError. Other possible values are 'ignore', 'replace' and\n\
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
88 'xmlcharrefreplace' as well as any other name registered with\n\
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
89 codecs.register_error that can handle ValueErrors.");
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
90
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
91 static PyObject *
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
92 codec_encode(PyObject *self, PyObject *args)
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
93 {
32602
ab979ff8b02a Change some declarations from ``char *`` to ``const char *``. Also added
Brett Cannon <bcannon@gmail.com>
parents: 32572
diff changeset
94 const char *encoding = NULL;
ab979ff8b02a Change some declarations from ``char *`` to ``const char *``. Also added
Brett Cannon <bcannon@gmail.com>
parents: 32572
diff changeset
95 const char *errors = NULL;
32572
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
96 PyObject *v;
35825
cc17a582518a Whitespace cleanup.
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
97
32572
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
98 if (!PyArg_ParseTuple(args, "O|ss:encode", &v, &encoding, &errors))
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
99 return NULL;
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
100
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
101 if (encoding == NULL)
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
102 encoding = PyUnicode_GetDefaultEncoding();
32572
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
103
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
104 /* Encode via the codec registry */
38669
19fd6141940d Partially merge trunk into p3yk. The removal of Mac/Tools is confusing svn
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
105 return PyCodec_Encode(v, encoding, errors);
32572
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
106 }
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
107
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
108 PyDoc_STRVAR(decode__doc__,
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
109 "decode(obj, [encoding[,errors]]) -> object\n\
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
110 \n\
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
111 Decodes obj using the codec registered for encoding. encoding defaults\n\
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
112 to the default encoding. errors may be given to set a different error\n\
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
113 handling scheme. Default is 'strict' meaning that encoding errors raise\n\
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
114 a ValueError. Other possible values are 'ignore' and 'replace'\n\
49493
bdf4da5d04ad merge from trunk
Benjamin Peterson <benjamin@python.org>
parents: 48683
diff changeset
115 as well as any other name registered with codecs.register_error that is\n\
32572
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
116 able to handle ValueErrors.");
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
117
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
118 static PyObject *
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
119 codec_decode(PyObject *self, PyObject *args)
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
120 {
32602
ab979ff8b02a Change some declarations from ``char *`` to ``const char *``. Also added
Brett Cannon <bcannon@gmail.com>
parents: 32572
diff changeset
121 const char *encoding = NULL;
ab979ff8b02a Change some declarations from ``char *`` to ``const char *``. Also added
Brett Cannon <bcannon@gmail.com>
parents: 32572
diff changeset
122 const char *errors = NULL;
32572
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
123 PyObject *v;
35825
cc17a582518a Whitespace cleanup.
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
124
32572
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
125 if (!PyArg_ParseTuple(args, "O|ss:decode", &v, &encoding, &errors))
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
126 return NULL;
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
127
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
128 if (encoding == NULL)
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
129 encoding = PyUnicode_GetDefaultEncoding();
32572
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
130
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
131 /* Decode via the codec registry */
38669
19fd6141940d Partially merge trunk into p3yk. The removal of Mac/Tools is confusing svn
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
132 return PyCodec_Decode(v, encoding, errors);
32572
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
133 }
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
134
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
135 /* --- Helpers ------------------------------------------------------------ */
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
136
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
137 static
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
138 PyObject *codec_tuple(PyObject *unicode,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
139 Py_ssize_t len)
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
140 {
38669
19fd6141940d Partially merge trunk into p3yk. The removal of Mac/Tools is confusing svn
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
141 PyObject *v;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
142 if (unicode == NULL)
38669
19fd6141940d Partially merge trunk into p3yk. The removal of Mac/Tools is confusing svn
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
143 return NULL;
19fd6141940d Partially merge trunk into p3yk. The removal of Mac/Tools is confusing svn
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
144 v = Py_BuildValue("On", unicode, len);
19fd6141940d Partially merge trunk into p3yk. The removal of Mac/Tools is confusing svn
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
145 Py_DECREF(unicode);
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
146 return v;
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
147 }
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
148
24808
ab63af3ec349 Patch #505705: Remove eval in pickle and cPickle.
Martin v. Löwis <martin@v.loewis.de>
parents: 24471
diff changeset
149 /* --- String codecs ------------------------------------------------------ */
ab63af3ec349 Patch #505705: Remove eval in pickle and cPickle.
Martin v. Löwis <martin@v.loewis.de>
parents: 24471
diff changeset
150 static PyObject *
ab63af3ec349 Patch #505705: Remove eval in pickle and cPickle.
Martin v. Löwis <martin@v.loewis.de>
parents: 24471
diff changeset
151 escape_decode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
152 PyObject *args)
24808
ab63af3ec349 Patch #505705: Remove eval in pickle and cPickle.
Martin v. Löwis <martin@v.loewis.de>
parents: 24471
diff changeset
153 {
ab63af3ec349 Patch #505705: Remove eval in pickle and cPickle.
Martin v. Löwis <martin@v.loewis.de>
parents: 24471
diff changeset
154 const char *errors = NULL;
ab63af3ec349 Patch #505705: Remove eval in pickle and cPickle.
Martin v. Löwis <martin@v.loewis.de>
parents: 24471
diff changeset
155 const char *data;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 35825
diff changeset
156 Py_ssize_t size;
35825
cc17a582518a Whitespace cleanup.
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
157
24808
ab63af3ec349 Patch #505705: Remove eval in pickle and cPickle.
Martin v. Löwis <martin@v.loewis.de>
parents: 24471
diff changeset
158 if (!PyArg_ParseTuple(args, "s#|z:escape_decode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
159 &data, &size, &errors))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
160 return NULL;
47301
23ab586c427a Renamed PyString to PyBytes
Christian Heimes <christian@cheimes.de>
parents: 43933
diff changeset
161 return codec_tuple(PyBytes_DecodeEscape(data, size, errors, 0, NULL),
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
162 size);
24808
ab63af3ec349 Patch #505705: Remove eval in pickle and cPickle.
Martin v. Löwis <martin@v.loewis.de>
parents: 24471
diff changeset
163 }
ab63af3ec349 Patch #505705: Remove eval in pickle and cPickle.
Martin v. Löwis <martin@v.loewis.de>
parents: 24471
diff changeset
164
ab63af3ec349 Patch #505705: Remove eval in pickle and cPickle.
Martin v. Löwis <martin@v.loewis.de>
parents: 24471
diff changeset
165 static PyObject *
ab63af3ec349 Patch #505705: Remove eval in pickle and cPickle.
Martin v. Löwis <martin@v.loewis.de>
parents: 24471
diff changeset
166 escape_encode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
167 PyObject *args)
24808
ab63af3ec349 Patch #505705: Remove eval in pickle and cPickle.
Martin v. Löwis <martin@v.loewis.de>
parents: 24471
diff changeset
168 {
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
169 PyObject *str;
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
170 Py_ssize_t size;
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
171 Py_ssize_t newsize;
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
172 const char *errors = NULL;
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
173 PyObject *v;
24808
ab63af3ec349 Patch #505705: Remove eval in pickle and cPickle.
Martin v. Löwis <martin@v.loewis.de>
parents: 24471
diff changeset
174
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
175 if (!PyArg_ParseTuple(args, "O!|z:escape_encode",
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
176 &PyBytes_Type, &str, &errors))
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
177 return NULL;
24808
ab63af3ec349 Patch #505705: Remove eval in pickle and cPickle.
Martin v. Löwis <martin@v.loewis.de>
parents: 24471
diff changeset
178
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
179 size = PyBytes_GET_SIZE(str);
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
180 newsize = 4*size;
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
181 if (newsize > PY_SSIZE_T_MAX || newsize / 4 != size) {
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
182 PyErr_SetString(PyExc_OverflowError,
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
183 "string is too large to encode");
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
184 return NULL;
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
185 }
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
186 v = PyBytes_FromStringAndSize(NULL, newsize);
41769
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41587
diff changeset
187
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
188 if (v == NULL) {
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
189 return NULL;
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
190 }
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
191 else {
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
192 register Py_ssize_t i;
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
193 register char c;
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
194 register char *p = PyBytes_AS_STRING(v);
24808
ab63af3ec349 Patch #505705: Remove eval in pickle and cPickle.
Martin v. Löwis <martin@v.loewis.de>
parents: 24471
diff changeset
195
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
196 for (i = 0; i < size; i++) {
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
197 /* There's at least enough room for a hex escape */
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
198 assert(newsize - (p - PyBytes_AS_STRING(v)) >= 4);
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
199 c = PyBytes_AS_STRING(str)[i];
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
200 if (c == '\'' || c == '\\')
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
201 *p++ = '\\', *p++ = c;
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
202 else if (c == '\t')
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
203 *p++ = '\\', *p++ = 't';
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
204 else if (c == '\n')
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
205 *p++ = '\\', *p++ = 'n';
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
206 else if (c == '\r')
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
207 *p++ = '\\', *p++ = 'r';
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
208 else if (c < ' ' || c >= 0x7f) {
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
209 *p++ = '\\';
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
210 *p++ = 'x';
72918
d76338eacf7c Issue #13088: Add shared Py_hexdigits constant to format a number into base 16
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72475
diff changeset
211 *p++ = Py_hexdigits[(c & 0xf0) >> 4];
d76338eacf7c Issue #13088: Add shared Py_hexdigits constant to format a number into base 16
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72475
diff changeset
212 *p++ = Py_hexdigits[c & 0xf];
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
213 }
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
214 else
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
215 *p++ = c;
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
216 }
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
217 *p = '\0';
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
218 if (_PyBytes_Resize(&v, (p - PyBytes_AS_STRING(v)))) {
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
219 return NULL;
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
220 }
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
221 }
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
222
62215
f33e464a9a60 Merged revisions 79780 via svnmerge from
Philip Jenvey <pjenvey@underboss.org>
parents: 62211
diff changeset
223 return codec_tuple(v, size);
24808
ab63af3ec349 Patch #505705: Remove eval in pickle and cPickle.
Martin v. Löwis <martin@v.loewis.de>
parents: 24471
diff changeset
224 }
ab63af3ec349 Patch #505705: Remove eval in pickle and cPickle.
Martin v. Löwis <martin@v.loewis.de>
parents: 24471
diff changeset
225
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
226 /* --- Decoder ------------------------------------------------------------ */
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
227
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
228 static PyObject *
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
229 unicode_internal_decode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
230 PyObject *args)
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
231 {
14733
814f421e0326 Added a true unicode_internal_encode function and fixed the
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
232 PyObject *obj;
814f421e0326 Added a true unicode_internal_encode function and fixed the
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
233 const char *errors = NULL;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
234 const char *data;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 35825
diff changeset
235 Py_ssize_t size;
35825
cc17a582518a Whitespace cleanup.
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
236
14733
814f421e0326 Added a true unicode_internal_encode function and fixed the
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
237 if (!PyArg_ParseTuple(args, "O|z:unicode_internal_decode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
238 &obj, &errors))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
239 return NULL;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
240
27680
b19aaaeb6d64 Fix for [ 543344 ] Interpreter crashes when recoding; suggested
Marc-André Lemburg <mal@egenix.com>
parents: 25974
diff changeset
241 if (PyUnicode_Check(obj)) {
73476
b3e532c1485e Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents: 73302
diff changeset
242 if (PyUnicode_READY(obj) < 0)
b3e532c1485e Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents: 73302
diff changeset
243 return NULL;
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
244 Py_INCREF(obj);
73476
b3e532c1485e Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents: 73302
diff changeset
245 return codec_tuple(obj, PyUnicode_GET_LENGTH(obj));
27680
b19aaaeb6d64 Fix for [ 543344 ] Interpreter crashes when recoding; suggested
Marc-André Lemburg <mal@egenix.com>
parents: 25974
diff changeset
246 }
14733
814f421e0326 Added a true unicode_internal_encode function and fixed the
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
247 else {
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
248 if (PyObject_AsReadBuffer(obj, (const void **)&data, &size))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
249 return NULL;
35485
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34707
diff changeset
250
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
251 return codec_tuple(_PyUnicode_DecodeUnicodeInternal(data, size, errors),
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
252 size);
14733
814f421e0326 Added a true unicode_internal_encode function and fixed the
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
253 }
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
254 }
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
255
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
256 static PyObject *
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19343
diff changeset
257 utf_7_decode(PyObject *self,
43933
9c4bf18e5d30 Merged revisions 59056-59076 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents: 43777
diff changeset
258 PyObject *args)
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19343
diff changeset
259 {
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
260 Py_buffer pbuf;
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19343
diff changeset
261 const char *errors = NULL;
43933
9c4bf18e5d30 Merged revisions 59056-59076 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents: 43777
diff changeset
262 int final = 0;
9c4bf18e5d30 Merged revisions 59056-59076 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents: 43777
diff changeset
263 Py_ssize_t consumed;
9c4bf18e5d30 Merged revisions 59056-59076 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents: 43777
diff changeset
264 PyObject *decoded = NULL;
35825
cc17a582518a Whitespace cleanup.
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
265
50975
ec6f29608bf0 Merged revisions 68855 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 49493
diff changeset
266 if (!PyArg_ParseTuple(args, "y*|zi:utf_7_decode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
267 &pbuf, &errors, &final))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
268 return NULL;
48683
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
269 consumed = pbuf.len;
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19343
diff changeset
270
48683
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
271 decoded = PyUnicode_DecodeUTF7Stateful(pbuf.buf, pbuf.len, errors,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
272 final ? NULL : &consumed);
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
273 PyBuffer_Release(&pbuf);
43933
9c4bf18e5d30 Merged revisions 59056-59076 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents: 43777
diff changeset
274 if (decoded == NULL)
9c4bf18e5d30 Merged revisions 59056-59076 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents: 43777
diff changeset
275 return NULL;
9c4bf18e5d30 Merged revisions 59056-59076 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents: 43777
diff changeset
276 return codec_tuple(decoded, consumed);
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19343
diff changeset
277 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19343
diff changeset
278
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19343
diff changeset
279 static PyObject *
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
280 utf_8_decode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
281 PyObject *args)
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
282 {
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
283 Py_buffer pbuf;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
284 const char *errors = NULL;
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 32602
diff changeset
285 int final = 0;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 35825
diff changeset
286 Py_ssize_t consumed;
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 32602
diff changeset
287 PyObject *decoded = NULL;
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 32602
diff changeset
288
50975
ec6f29608bf0 Merged revisions 68855 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 49493
diff changeset
289 if (!PyArg_ParseTuple(args, "y*|zi:utf_8_decode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
290 &pbuf, &errors, &final))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
291 return NULL;
48683
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
292 consumed = pbuf.len;
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
293
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
294 decoded = PyUnicode_DecodeUTF8Stateful(pbuf.buf, pbuf.len, errors,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
295 final ? NULL : &consumed);
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
296 PyBuffer_Release(&pbuf);
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 32602
diff changeset
297 if (decoded == NULL)
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
298 return NULL;
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 32602
diff changeset
299 return codec_tuple(decoded, consumed);
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
300 }
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
301
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
302 static PyObject *
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
303 utf_16_decode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
304 PyObject *args)
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
305 {
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
306 Py_buffer pbuf;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
307 const char *errors = NULL;
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
308 int byteorder = 0;
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 32602
diff changeset
309 int final = 0;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 35825
diff changeset
310 Py_ssize_t consumed;
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 32602
diff changeset
311 PyObject *decoded;
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 32602
diff changeset
312
50975
ec6f29608bf0 Merged revisions 68855 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 49493
diff changeset
313 if (!PyArg_ParseTuple(args, "y*|zi:utf_16_decode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
314 &pbuf, &errors, &final))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
315 return NULL;
48683
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
316 consumed = pbuf.len; /* This is overwritten unless final is true. */
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
317 decoded = PyUnicode_DecodeUTF16Stateful(pbuf.buf, pbuf.len, errors,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
318 &byteorder, final ? NULL : &consumed);
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
319 PyBuffer_Release(&pbuf);
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 32602
diff changeset
320 if (decoded == NULL)
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
321 return NULL;
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 32602
diff changeset
322 return codec_tuple(decoded, consumed);
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
323 }
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
324
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
325 static PyObject *
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
326 utf_16_le_decode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
327 PyObject *args)
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
328 {
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
329 Py_buffer pbuf;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
330 const char *errors = NULL;
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
331 int byteorder = -1;
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 32602
diff changeset
332 int final = 0;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 35825
diff changeset
333 Py_ssize_t consumed;
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 32602
diff changeset
334 PyObject *decoded = NULL;
35825
cc17a582518a Whitespace cleanup.
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
335
50975
ec6f29608bf0 Merged revisions 68855 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 49493
diff changeset
336 if (!PyArg_ParseTuple(args, "y*|zi:utf_16_le_decode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
337 &pbuf, &errors, &final))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
338 return NULL;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 35825
diff changeset
339
48683
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
340 consumed = pbuf.len; /* This is overwritten unless final is true. */
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
341 decoded = PyUnicode_DecodeUTF16Stateful(pbuf.buf, pbuf.len, errors,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
342 &byteorder, final ? NULL : &consumed);
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
343 PyBuffer_Release(&pbuf);
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 32602
diff changeset
344 if (decoded == NULL)
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
345 return NULL;
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 32602
diff changeset
346 return codec_tuple(decoded, consumed);
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
347 }
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
348
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
349 static PyObject *
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
350 utf_16_be_decode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
351 PyObject *args)
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
352 {
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
353 Py_buffer pbuf;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
354 const char *errors = NULL;
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
355 int byteorder = 1;
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 32602
diff changeset
356 int final = 0;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 35825
diff changeset
357 Py_ssize_t consumed;
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 32602
diff changeset
358 PyObject *decoded = NULL;
35825
cc17a582518a Whitespace cleanup.
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
359
50975
ec6f29608bf0 Merged revisions 68855 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 49493
diff changeset
360 if (!PyArg_ParseTuple(args, "y*|zi:utf_16_be_decode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
361 &pbuf, &errors, &final))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
362 return NULL;
48683
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
363
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
364 consumed = pbuf.len; /* This is overwritten unless final is true. */
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
365 decoded = PyUnicode_DecodeUTF16Stateful(pbuf.buf, pbuf.len, errors,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
366 &byteorder, final ? NULL : &consumed);
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
367 PyBuffer_Release(&pbuf);
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 32602
diff changeset
368 if (decoded == NULL)
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
369 return NULL;
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 32602
diff changeset
370 return codec_tuple(decoded, consumed);
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
371 }
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
372
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
373 /* This non-standard version also provides access to the byteorder
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
374 parameter of the builtin UTF-16 codec.
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
375
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
376 It returns a tuple (unicode, bytesread, byteorder) with byteorder
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
377 being the value in effect at the end of data.
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
378
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
379 */
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
380
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
381 static PyObject *
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
382 utf_16_ex_decode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
383 PyObject *args)
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
384 {
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
385 Py_buffer pbuf;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
386 const char *errors = NULL;
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
387 int byteorder = 0;
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
388 PyObject *unicode, *tuple;
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 32602
diff changeset
389 int final = 0;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 35825
diff changeset
390 Py_ssize_t consumed;
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 32602
diff changeset
391
50975
ec6f29608bf0 Merged revisions 68855 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 49493
diff changeset
392 if (!PyArg_ParseTuple(args, "y*|zii:utf_16_ex_decode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
393 &pbuf, &errors, &byteorder, &final))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
394 return NULL;
48683
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
395 consumed = pbuf.len; /* This is overwritten unless final is true. */
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
396 unicode = PyUnicode_DecodeUTF16Stateful(pbuf.buf, pbuf.len, errors,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
397 &byteorder, final ? NULL : &consumed);
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
398 PyBuffer_Release(&pbuf);
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
399 if (unicode == NULL)
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
400 return NULL;
38669
19fd6141940d Partially merge trunk into p3yk. The removal of Mac/Tools is confusing svn
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
401 tuple = Py_BuildValue("Oni", unicode, consumed, byteorder);
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
402 Py_DECREF(unicode);
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
403 return tuple;
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
404 }
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
405
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
406 static PyObject *
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
407 utf_32_decode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
408 PyObject *args)
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
409 {
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
410 Py_buffer pbuf;
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
411 const char *errors = NULL;
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
412 int byteorder = 0;
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
413 int final = 0;
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
414 Py_ssize_t consumed;
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
415 PyObject *decoded;
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
416
50975
ec6f29608bf0 Merged revisions 68855 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 49493
diff changeset
417 if (!PyArg_ParseTuple(args, "y*|zi:utf_32_decode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
418 &pbuf, &errors, &final))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
419 return NULL;
48683
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
420 consumed = pbuf.len; /* This is overwritten unless final is true. */
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
421 decoded = PyUnicode_DecodeUTF32Stateful(pbuf.buf, pbuf.len, errors,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
422 &byteorder, final ? NULL : &consumed);
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
423 PyBuffer_Release(&pbuf);
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
424 if (decoded == NULL)
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
425 return NULL;
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
426 return codec_tuple(decoded, consumed);
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
427 }
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
428
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
429 static PyObject *
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
430 utf_32_le_decode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
431 PyObject *args)
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
432 {
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
433 Py_buffer pbuf;
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
434 const char *errors = NULL;
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
435 int byteorder = -1;
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
436 int final = 0;
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
437 Py_ssize_t consumed;
48683
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
438 PyObject *decoded;
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
439
50975
ec6f29608bf0 Merged revisions 68855 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 49493
diff changeset
440 if (!PyArg_ParseTuple(args, "y*|zi:utf_32_le_decode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
441 &pbuf, &errors, &final))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
442 return NULL;
48683
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
443 consumed = pbuf.len; /* This is overwritten unless final is true. */
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
444 decoded = PyUnicode_DecodeUTF32Stateful(pbuf.buf, pbuf.len, errors,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
445 &byteorder, final ? NULL : &consumed);
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
446 PyBuffer_Release(&pbuf);
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
447 if (decoded == NULL)
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
448 return NULL;
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
449 return codec_tuple(decoded, consumed);
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
450 }
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
451
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
452 static PyObject *
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
453 utf_32_be_decode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
454 PyObject *args)
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
455 {
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
456 Py_buffer pbuf;
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
457 const char *errors = NULL;
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
458 int byteorder = 1;
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
459 int final = 0;
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
460 Py_ssize_t consumed;
48683
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
461 PyObject *decoded;
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
462
50975
ec6f29608bf0 Merged revisions 68855 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 49493
diff changeset
463 if (!PyArg_ParseTuple(args, "y*|zi:utf_32_be_decode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
464 &pbuf, &errors, &final))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
465 return NULL;
48683
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
466 consumed = pbuf.len; /* This is overwritten unless final is true. */
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
467 decoded = PyUnicode_DecodeUTF32Stateful(pbuf.buf, pbuf.len, errors,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
468 &byteorder, final ? NULL : &consumed);
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
469 PyBuffer_Release(&pbuf);
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
470 if (decoded == NULL)
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
471 return NULL;
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
472 return codec_tuple(decoded, consumed);
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
473 }
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
474
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
475 /* This non-standard version also provides access to the byteorder
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
476 parameter of the builtin UTF-32 codec.
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
477
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
478 It returns a tuple (unicode, bytesread, byteorder) with byteorder
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
479 being the value in effect at the end of data.
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
480
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
481 */
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
482
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
483 static PyObject *
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
484 utf_32_ex_decode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
485 PyObject *args)
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
486 {
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
487 Py_buffer pbuf;
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
488 const char *errors = NULL;
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
489 int byteorder = 0;
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
490 PyObject *unicode, *tuple;
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
491 int final = 0;
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
492 Py_ssize_t consumed;
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
493
50975
ec6f29608bf0 Merged revisions 68855 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 49493
diff changeset
494 if (!PyArg_ParseTuple(args, "y*|zii:utf_32_ex_decode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
495 &pbuf, &errors, &byteorder, &final))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
496 return NULL;
48683
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
497 consumed = pbuf.len; /* This is overwritten unless final is true. */
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
498 unicode = PyUnicode_DecodeUTF32Stateful(pbuf.buf, pbuf.len, errors,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
499 &byteorder, final ? NULL : &consumed);
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
500 PyBuffer_Release(&pbuf);
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
501 if (unicode == NULL)
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
502 return NULL;
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
503 tuple = Py_BuildValue("Oni", unicode, consumed, byteorder);
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
504 Py_DECREF(unicode);
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
505 return tuple;
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
506 }
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
507
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
508 static PyObject *
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
509 unicode_escape_decode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
510 PyObject *args)
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
511 {
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
512 Py_buffer pbuf;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
513 const char *errors = NULL;
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
514 PyObject *unicode;
35825
cc17a582518a Whitespace cleanup.
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
515
48683
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
516 if (!PyArg_ParseTuple(args, "s*|z:unicode_escape_decode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
517 &pbuf, &errors))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
518 return NULL;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
519
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
520 unicode = PyUnicode_DecodeUnicodeEscape(pbuf.buf, pbuf.len, errors);
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
521 PyBuffer_Release(&pbuf);
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
522 return codec_tuple(unicode, pbuf.len);
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
523 }
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
524
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
525 static PyObject *
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
526 raw_unicode_escape_decode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
527 PyObject *args)
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
528 {
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
529 Py_buffer pbuf;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
530 const char *errors = NULL;
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
531 PyObject *unicode;
35825
cc17a582518a Whitespace cleanup.
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
532
48683
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
533 if (!PyArg_ParseTuple(args, "s*|z:raw_unicode_escape_decode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
534 &pbuf, &errors))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
535 return NULL;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
536
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
537 unicode = PyUnicode_DecodeRawUnicodeEscape(pbuf.buf, pbuf.len, errors);
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
538 PyBuffer_Release(&pbuf);
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
539 return codec_tuple(unicode, pbuf.len);
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
540 }
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
541
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
542 static PyObject *
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
543 latin_1_decode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
544 PyObject *args)
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
545 {
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
546 Py_buffer pbuf;
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
547 PyObject *unicode;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
548 const char *errors = NULL;
35825
cc17a582518a Whitespace cleanup.
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
549
50975
ec6f29608bf0 Merged revisions 68855 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 49493
diff changeset
550 if (!PyArg_ParseTuple(args, "y*|z:latin_1_decode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
551 &pbuf, &errors))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
552 return NULL;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
553
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
554 unicode = PyUnicode_DecodeLatin1(pbuf.buf, pbuf.len, errors);
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
555 PyBuffer_Release(&pbuf);
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
556 return codec_tuple(unicode, pbuf.len);
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
557 }
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
558
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
559 static PyObject *
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
560 ascii_decode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
561 PyObject *args)
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
562 {
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
563 Py_buffer pbuf;
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
564 PyObject *unicode;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
565 const char *errors = NULL;
35825
cc17a582518a Whitespace cleanup.
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
566
50975
ec6f29608bf0 Merged revisions 68855 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 49493
diff changeset
567 if (!PyArg_ParseTuple(args, "y*|z:ascii_decode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
568 &pbuf, &errors))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
569 return NULL;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
570
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
571 unicode = PyUnicode_DecodeASCII(pbuf.buf, pbuf.len, errors);
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
572 PyBuffer_Release(&pbuf);
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
573 return codec_tuple(unicode, pbuf.len);
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
574 }
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
575
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
576 static PyObject *
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
577 charmap_decode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
578 PyObject *args)
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
579 {
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
580 Py_buffer pbuf;
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
581 PyObject *unicode;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
582 const char *errors = NULL;
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
583 PyObject *mapping = NULL;
35825
cc17a582518a Whitespace cleanup.
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
584
50975
ec6f29608bf0 Merged revisions 68855 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 49493
diff changeset
585 if (!PyArg_ParseTuple(args, "y*|zO:charmap_decode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
586 &pbuf, &errors, &mapping))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
587 return NULL;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
588 if (mapping == Py_None)
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
589 mapping = NULL;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
590
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
591 unicode = PyUnicode_DecodeCharmap(pbuf.buf, pbuf.len, mapping, errors);
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
592 PyBuffer_Release(&pbuf);
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
593 return codec_tuple(unicode, pbuf.len);
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
594 }
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
595
71193
13e6d3cb2ecd Issue #9642: Uniformize the tests on the availability of the mbcs codec
Victor Stinner <victor.stinner@haypocalc.com>
parents: 68257
diff changeset
596 #ifdef HAVE_MBCS
12074
65f35b73e350 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 11980
diff changeset
597
65f35b73e350 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 11980
diff changeset
598 static PyObject *
65f35b73e350 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 11980
diff changeset
599 mbcs_decode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
600 PyObject *args)
12074
65f35b73e350 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 11980
diff changeset
601 {
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
602 Py_buffer pbuf;
12074
65f35b73e350 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 11980
diff changeset
603 const char *errors = NULL;
39442
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 38673
diff changeset
604 int final = 0;
48683
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
605 Py_ssize_t consumed;
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
606 PyObject *decoded = NULL;
12074
65f35b73e350 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 11980
diff changeset
607
50975
ec6f29608bf0 Merged revisions 68855 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 49493
diff changeset
608 if (!PyArg_ParseTuple(args, "y*|zi:mbcs_decode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
609 &pbuf, &errors, &final))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
610 return NULL;
48683
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
611 consumed = pbuf.len;
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
612
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
613 decoded = PyUnicode_DecodeMBCSStateful(pbuf.buf, pbuf.len, errors,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
614 final ? NULL : &consumed);
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
615 PyBuffer_Release(&pbuf);
48683
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
616 if (decoded == NULL)
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
617 return NULL;
48683
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
618 return codec_tuple(decoded, consumed);
12074
65f35b73e350 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 11980
diff changeset
619 }
65f35b73e350 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 11980
diff changeset
620
72976
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
621 static PyObject *
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
622 code_page_decode(PyObject *self,
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
623 PyObject *args)
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
624 {
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
625 Py_buffer pbuf;
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
626 const char *errors = NULL;
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
627 int final = 0;
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
628 Py_ssize_t consumed;
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
629 PyObject *decoded = NULL;
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
630 int code_page;
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
631
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
632 if (!PyArg_ParseTuple(args, "iy*|zi:code_page_decode",
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
633 &code_page, &pbuf, &errors, &final))
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
634 return NULL;
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
635 consumed = pbuf.len;
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
636
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
637 decoded = PyUnicode_DecodeCodePageStateful(code_page,
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
638 pbuf.buf, pbuf.len, errors,
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
639 final ? NULL : &consumed);
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
640 PyBuffer_Release(&pbuf);
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
641 if (decoded == NULL)
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
642 return NULL;
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
643 return codec_tuple(decoded, consumed);
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
644 }
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
645
71193
13e6d3cb2ecd Issue #9642: Uniformize the tests on the availability of the mbcs codec
Victor Stinner <victor.stinner@haypocalc.com>
parents: 68257
diff changeset
646 #endif /* HAVE_MBCS */
12074
65f35b73e350 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 11980
diff changeset
647
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
648 /* --- Encoder ------------------------------------------------------------ */
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
649
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
650 static PyObject *
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
651 readbuffer_encode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
652 PyObject *args)
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
653 {
48683
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
654 Py_buffer pdata;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
655 const char *data;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 35825
diff changeset
656 Py_ssize_t size;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
657 const char *errors = NULL;
48683
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
658 PyObject *result;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
659
48683
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
660 if (!PyArg_ParseTuple(args, "s*|z:readbuffer_encode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
661 &pdata, &errors))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
662 return NULL;
48683
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
663 data = pdata.buf;
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
664 size = pdata.len;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
665
48683
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
666 result = PyBytes_FromStringAndSize(data, size);
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
667 PyBuffer_Release(&pdata);
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
668 return codec_tuple(result, size);
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
669 }
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
670
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
671 static PyObject *
14733
814f421e0326 Added a true unicode_internal_encode function and fixed the
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
672 unicode_internal_encode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
673 PyObject *args)
14733
814f421e0326 Added a true unicode_internal_encode function and fixed the
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
674 {
814f421e0326 Added a true unicode_internal_encode function and fixed the
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
675 PyObject *obj;
814f421e0326 Added a true unicode_internal_encode function and fixed the
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
676 const char *errors = NULL;
814f421e0326 Added a true unicode_internal_encode function and fixed the
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
677 const char *data;
73486
6e660d9fd509 Fix and deprecated the unicode_internal codec
Victor Stinner <victor.stinner@haypocalc.com>
parents: 73479
diff changeset
678 Py_ssize_t len, size;
35825
cc17a582518a Whitespace cleanup.
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
679
14733
814f421e0326 Added a true unicode_internal_encode function and fixed the
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
680 if (!PyArg_ParseTuple(args, "O|z:unicode_internal_encode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
681 &obj, &errors))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
682 return NULL;
14733
814f421e0326 Added a true unicode_internal_encode function and fixed the
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
683
814f421e0326 Added a true unicode_internal_encode function and fixed the
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
684 if (PyUnicode_Check(obj)) {
73486
6e660d9fd509 Fix and deprecated the unicode_internal codec
Victor Stinner <victor.stinner@haypocalc.com>
parents: 73479
diff changeset
685 Py_UNICODE *u;
6e660d9fd509 Fix and deprecated the unicode_internal codec
Victor Stinner <victor.stinner@haypocalc.com>
parents: 73479
diff changeset
686
73476
b3e532c1485e Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents: 73302
diff changeset
687 if (PyUnicode_READY(obj) < 0)
b3e532c1485e Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents: 73302
diff changeset
688 return NULL;
73486
6e660d9fd509 Fix and deprecated the unicode_internal codec
Victor Stinner <victor.stinner@haypocalc.com>
parents: 73479
diff changeset
689
6e660d9fd509 Fix and deprecated the unicode_internal codec
Victor Stinner <victor.stinner@haypocalc.com>
parents: 73479
diff changeset
690 if (PyErr_WarnEx(PyExc_DeprecationWarning,
6e660d9fd509 Fix and deprecated the unicode_internal codec
Victor Stinner <victor.stinner@haypocalc.com>
parents: 73479
diff changeset
691 "unicode_internal codecs has been deprecated",
6e660d9fd509 Fix and deprecated the unicode_internal codec
Victor Stinner <victor.stinner@haypocalc.com>
parents: 73479
diff changeset
692 1))
6e660d9fd509 Fix and deprecated the unicode_internal codec
Victor Stinner <victor.stinner@haypocalc.com>
parents: 73479
diff changeset
693 return NULL;
6e660d9fd509 Fix and deprecated the unicode_internal codec
Victor Stinner <victor.stinner@haypocalc.com>
parents: 73479
diff changeset
694
6e660d9fd509 Fix and deprecated the unicode_internal codec
Victor Stinner <victor.stinner@haypocalc.com>
parents: 73479
diff changeset
695 u = PyUnicode_AsUnicodeAndSize(obj, &len);
6e660d9fd509 Fix and deprecated the unicode_internal codec
Victor Stinner <victor.stinner@haypocalc.com>
parents: 73479
diff changeset
696 if (u == NULL)
6e660d9fd509 Fix and deprecated the unicode_internal codec
Victor Stinner <victor.stinner@haypocalc.com>
parents: 73479
diff changeset
697 return NULL;
6e660d9fd509 Fix and deprecated the unicode_internal codec
Victor Stinner <victor.stinner@haypocalc.com>
parents: 73479
diff changeset
698 if (len > PY_SSIZE_T_MAX / sizeof(Py_UNICODE))
6e660d9fd509 Fix and deprecated the unicode_internal codec
Victor Stinner <victor.stinner@haypocalc.com>
parents: 73479
diff changeset
699 return PyErr_NoMemory();
6e660d9fd509 Fix and deprecated the unicode_internal codec
Victor Stinner <victor.stinner@haypocalc.com>
parents: 73479
diff changeset
700 size = len * sizeof(Py_UNICODE);
6e660d9fd509 Fix and deprecated the unicode_internal codec
Victor Stinner <victor.stinner@haypocalc.com>
parents: 73479
diff changeset
701 return codec_tuple(PyBytes_FromStringAndSize((const char*)u, size),
73476
b3e532c1485e Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents: 73302
diff changeset
702 PyUnicode_GET_LENGTH(obj));
14733
814f421e0326 Added a true unicode_internal_encode function and fixed the
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
703 }
814f421e0326 Added a true unicode_internal_encode function and fixed the
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
704 else {
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
705 if (PyObject_AsReadBuffer(obj, (const void **)&data, &size))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
706 return NULL;
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
707 return codec_tuple(PyBytes_FromStringAndSize(data, size), size);
14733
814f421e0326 Added a true unicode_internal_encode function and fixed the
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
708 }
814f421e0326 Added a true unicode_internal_encode function and fixed the
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
709 }
814f421e0326 Added a true unicode_internal_encode function and fixed the
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
710
814f421e0326 Added a true unicode_internal_encode function and fixed the
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
711 static PyObject *
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19343
diff changeset
712 utf_7_encode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
713 PyObject *args)
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19343
diff changeset
714 {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19343
diff changeset
715 PyObject *str, *v;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19343
diff changeset
716 const char *errors = NULL;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19343
diff changeset
717
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19343
diff changeset
718 if (!PyArg_ParseTuple(args, "O|z:utf_7_encode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
719 &str, &errors))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
720 return NULL;
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19343
diff changeset
721
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19343
diff changeset
722 str = PyUnicode_FromObject(str);
73476
b3e532c1485e Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents: 73302
diff changeset
723 if (str == NULL || PyUnicode_READY(str) < 0)
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
724 return NULL;
73476
b3e532c1485e Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents: 73302
diff changeset
725 v = codec_tuple(_PyUnicode_EncodeUTF7(str, 0, 0, errors),
b3e532c1485e Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents: 73302
diff changeset
726 PyUnicode_GET_LENGTH(str));
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19343
diff changeset
727 Py_DECREF(str);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19343
diff changeset
728 return v;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19343
diff changeset
729 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19343
diff changeset
730
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19343
diff changeset
731 static PyObject *
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
732 utf_8_encode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
733 PyObject *args)
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
734 {
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
735 PyObject *str, *v;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
736 const char *errors = NULL;
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
737
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
738 if (!PyArg_ParseTuple(args, "O|z:utf_8_encode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
739 &str, &errors))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
740 return NULL;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
741
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
742 str = PyUnicode_FromObject(str);
72475
8beaa9a37387 Implement PEP 393.
Martin v. Löwis <martin@v.loewis.de>
parents: 71193
diff changeset
743 if (str == NULL || PyUnicode_READY(str) == -1)
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
744 return NULL;
72475
8beaa9a37387 Implement PEP 393.
Martin v. Löwis <martin@v.loewis.de>
parents: 71193
diff changeset
745 v = codec_tuple(PyUnicode_AsEncodedString(str, "utf-8", errors),
8beaa9a37387 Implement PEP 393.
Martin v. Löwis <martin@v.loewis.de>
parents: 71193
diff changeset
746 PyUnicode_GET_LENGTH(str));
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
747 Py_DECREF(str);
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
748 return v;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
749 }
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
750
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
751 /* This version provides access to the byteorder parameter of the
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
752 builtin UTF-16 codecs as optional third argument. It defaults to 0
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
753 which means: use the native byte order and prepend the data with a
35825
cc17a582518a Whitespace cleanup.
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
754 BOM mark.
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
755
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
756 */
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
757
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
758 static PyObject *
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
759 utf_16_encode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
760 PyObject *args)
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
761 {
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
762 PyObject *str, *v;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
763 const char *errors = NULL;
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
764 int byteorder = 0;
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
765
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
766 if (!PyArg_ParseTuple(args, "O|zi:utf_16_encode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
767 &str, &errors, &byteorder))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
768 return NULL;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
769
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
770 str = PyUnicode_FromObject(str);
73476
b3e532c1485e Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents: 73302
diff changeset
771 if (str == NULL || PyUnicode_READY(str) < 0)
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
772 return NULL;
73476
b3e532c1485e Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents: 73302
diff changeset
773 v = codec_tuple(_PyUnicode_EncodeUTF16(str, errors, byteorder),
b3e532c1485e Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents: 73302
diff changeset
774 PyUnicode_GET_LENGTH(str));
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
775 Py_DECREF(str);
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
776 return v;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
777 }
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
778
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
779 static PyObject *
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
780 utf_16_le_encode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
781 PyObject *args)
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
782 {
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
783 PyObject *str, *v;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
784 const char *errors = NULL;
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
785
18139
354a4df09e38 Fix for bug [ #433047 ] missing args to PyArg_ParseTuple
Marc-André Lemburg <mal@egenix.com>
parents: 14733
diff changeset
786 if (!PyArg_ParseTuple(args, "O|z:utf_16_le_encode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
787 &str, &errors))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
788 return NULL;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
789
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
790 str = PyUnicode_FromObject(str);
73476
b3e532c1485e Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents: 73302
diff changeset
791 if (str == NULL || PyUnicode_READY(str) < 0)
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
792 return NULL;
73476
b3e532c1485e Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents: 73302
diff changeset
793 v = codec_tuple(_PyUnicode_EncodeUTF16(str, errors, -1),
b3e532c1485e Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents: 73302
diff changeset
794 PyUnicode_GET_LENGTH(str));
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
795 Py_DECREF(str);
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
796 return v;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
797 }
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
798
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
799 static PyObject *
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
800 utf_16_be_encode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
801 PyObject *args)
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
802 {
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
803 PyObject *str, *v;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
804 const char *errors = NULL;
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
805
18139
354a4df09e38 Fix for bug [ #433047 ] missing args to PyArg_ParseTuple
Marc-André Lemburg <mal@egenix.com>
parents: 14733
diff changeset
806 if (!PyArg_ParseTuple(args, "O|z:utf_16_be_encode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
807 &str, &errors))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
808 return NULL;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
809
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
810 str = PyUnicode_FromObject(str);
73476
b3e532c1485e Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents: 73302
diff changeset
811 if (str == NULL || PyUnicode_READY(str) < 0)
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
812 return NULL;
73476
b3e532c1485e Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents: 73302
diff changeset
813 v = codec_tuple(_PyUnicode_EncodeUTF16(str, errors, +1),
b3e532c1485e Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents: 73302
diff changeset
814 PyUnicode_GET_LENGTH(str));
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
815 Py_DECREF(str);
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
816 return v;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
817 }
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
818
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
819 /* This version provides access to the byteorder parameter of the
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
820 builtin UTF-32 codecs as optional third argument. It defaults to 0
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
821 which means: use the native byte order and prepend the data with a
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
822 BOM mark.
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
823
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
824 */
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
825
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
826 static PyObject *
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
827 utf_32_encode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
828 PyObject *args)
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
829 {
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
830 PyObject *str, *v;
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
831 const char *errors = NULL;
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
832 int byteorder = 0;
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
833
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
834 if (!PyArg_ParseTuple(args, "O|zi:utf_32_encode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
835 &str, &errors, &byteorder))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
836 return NULL;
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
837
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
838 str = PyUnicode_FromObject(str);
73476
b3e532c1485e Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents: 73302
diff changeset
839 if (str == NULL || PyUnicode_READY(str) < 0)
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
840 return NULL;
73476
b3e532c1485e Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents: 73302
diff changeset
841 v = codec_tuple(_PyUnicode_EncodeUTF32(str, errors, byteorder),
b3e532c1485e Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents: 73302
diff changeset
842 PyUnicode_GET_LENGTH(str));
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
843 Py_DECREF(str);
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
844 return v;
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
845 }
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
846
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
847 static PyObject *
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
848 utf_32_le_encode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
849 PyObject *args)
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
850 {
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
851 PyObject *str, *v;
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
852 const char *errors = NULL;
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
853
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
854 if (!PyArg_ParseTuple(args, "O|z:utf_32_le_encode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
855 &str, &errors))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
856 return NULL;
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
857
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
858 str = PyUnicode_FromObject(str);
73476
b3e532c1485e Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents: 73302
diff changeset
859 if (str == NULL || PyUnicode_READY(str) < 0)
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
860 return NULL;
73476
b3e532c1485e Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents: 73302
diff changeset
861 v = codec_tuple(_PyUnicode_EncodeUTF32(str, errors, -1),
b3e532c1485e Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents: 73302
diff changeset
862 PyUnicode_GET_LENGTH(str));
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
863 Py_DECREF(str);
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
864 return v;
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
865 }
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
866
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
867 static PyObject *
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
868 utf_32_be_encode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
869 PyObject *args)
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
870 {
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
871 PyObject *str, *v;
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
872 const char *errors = NULL;
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
873
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
874 if (!PyArg_ParseTuple(args, "O|z:utf_32_be_encode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
875 &str, &errors))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
876 return NULL;
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
877
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
878 str = PyUnicode_FromObject(str);
73476
b3e532c1485e Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents: 73302
diff changeset
879 if (str == NULL || PyUnicode_READY(str) < 0)
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
880 return NULL;
73476
b3e532c1485e Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents: 73302
diff changeset
881 v = codec_tuple(_PyUnicode_EncodeUTF32(str, errors, +1),
b3e532c1485e Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents: 73302
diff changeset
882 PyUnicode_GET_LENGTH(str));
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
883 Py_DECREF(str);
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
884 return v;
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
885 }
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
886
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
887 static PyObject *
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
888 unicode_escape_encode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
889 PyObject *args)
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
890 {
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
891 PyObject *str, *v;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
892 const char *errors = NULL;
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
893
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
894 if (!PyArg_ParseTuple(args, "O|z:unicode_escape_encode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
895 &str, &errors))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
896 return NULL;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
897
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
898 str = PyUnicode_FromObject(str);
73476
b3e532c1485e Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents: 73302
diff changeset
899 if (str == NULL || PyUnicode_READY(str) < 0)
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
900 return NULL;
73476
b3e532c1485e Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents: 73302
diff changeset
901 v = codec_tuple(PyUnicode_AsUnicodeEscapeString(str),
b3e532c1485e Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents: 73302
diff changeset
902 PyUnicode_GET_LENGTH(str));
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
903 Py_DECREF(str);
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
904 return v;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
905 }
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
906
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
907 static PyObject *
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
908 raw_unicode_escape_encode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
909 PyObject *args)
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
910 {
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
911 PyObject *str, *v;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
912 const char *errors = NULL;
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
913
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
914 if (!PyArg_ParseTuple(args, "O|z:raw_unicode_escape_encode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
915 &str, &errors))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
916 return NULL;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
917
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
918 str = PyUnicode_FromObject(str);
73476
b3e532c1485e Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents: 73302
diff changeset
919 if (str == NULL || PyUnicode_READY(str) < 0)
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
920 return NULL;
73476
b3e532c1485e Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents: 73302
diff changeset
921 v = codec_tuple(PyUnicode_AsRawUnicodeEscapeString(str),
b3e532c1485e Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents: 73302
diff changeset
922 PyUnicode_GET_LENGTH(str));
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
923 Py_DECREF(str);
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
924 return v;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
925 }
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
926
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
927 static PyObject *
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
928 latin_1_encode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
929 PyObject *args)
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
930 {
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
931 PyObject *str, *v;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
932 const char *errors = NULL;
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
933
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
934 if (!PyArg_ParseTuple(args, "O|z:latin_1_encode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
935 &str, &errors))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
936 return NULL;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
937
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
938 str = PyUnicode_FromObject(str);
73476
b3e532c1485e Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents: 73302
diff changeset
939 if (str == NULL || PyUnicode_READY(str) < 0)
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
940 return NULL;
73476
b3e532c1485e Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents: 73302
diff changeset
941 v = codec_tuple(_PyUnicode_AsLatin1String(str, errors),
b3e532c1485e Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents: 73302
diff changeset
942 PyUnicode_GET_LENGTH(str));
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
943 Py_DECREF(str);
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
944 return v;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
945 }
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
946
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
947 static PyObject *
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
948 ascii_encode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
949 PyObject *args)
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
950 {
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
951 PyObject *str, *v;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
952 const char *errors = NULL;
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
953
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
954 if (!PyArg_ParseTuple(args, "O|z:ascii_encode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
955 &str, &errors))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
956 return NULL;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
957
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
958 str = PyUnicode_FromObject(str);
73476
b3e532c1485e Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents: 73302
diff changeset
959 if (str == NULL || PyUnicode_READY(str) < 0)
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
960 return NULL;
73476
b3e532c1485e Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents: 73302
diff changeset
961 v = codec_tuple(_PyUnicode_AsASCIIString(str, errors),
b3e532c1485e Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents: 73302
diff changeset
962 PyUnicode_GET_LENGTH(str));
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
963 Py_DECREF(str);
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
964 return v;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
965 }
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
966
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
967 static PyObject *
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
968 charmap_encode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
969 PyObject *args)
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
970 {
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
971 PyObject *str, *v;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
972 const char *errors = NULL;
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
973 PyObject *mapping = NULL;
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
974
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
975 if (!PyArg_ParseTuple(args, "O|zO:charmap_encode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
976 &str, &errors, &mapping))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
977 return NULL;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
978 if (mapping == Py_None)
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
979 mapping = NULL;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
980
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
981 str = PyUnicode_FromObject(str);
73476
b3e532c1485e Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents: 73302
diff changeset
982 if (str == NULL || PyUnicode_READY(str) < 0)
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
983 return NULL;
73302
295fdfd4f422 Port UCS1 and charmap codecs to new API.
Martin v. Löwis <martin@v.loewis.de>
parents: 72976
diff changeset
984 v = codec_tuple(_PyUnicode_EncodeCharmap(str, mapping, errors),
73476
b3e532c1485e Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents: 73302
diff changeset
985 PyUnicode_GET_LENGTH(str));
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
986 Py_DECREF(str);
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
987 return v;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
988 }
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
989
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38669
diff changeset
990 static PyObject*
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38669
diff changeset
991 charmap_build(PyObject *self, PyObject *args)
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38669
diff changeset
992 {
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38669
diff changeset
993 PyObject *map;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38669
diff changeset
994 if (!PyArg_ParseTuple(args, "U:charmap_build", &map))
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38669
diff changeset
995 return NULL;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38669
diff changeset
996 return PyUnicode_BuildEncodingMap(map);
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38669
diff changeset
997 }
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38669
diff changeset
998
71193
13e6d3cb2ecd Issue #9642: Uniformize the tests on the availability of the mbcs codec
Victor Stinner <victor.stinner@haypocalc.com>
parents: 68257
diff changeset
999 #ifdef HAVE_MBCS
12074
65f35b73e350 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 11980
diff changeset
1000
65f35b73e350 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 11980
diff changeset
1001 static PyObject *
65f35b73e350 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 11980
diff changeset
1002 mbcs_encode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1003 PyObject *args)
12074
65f35b73e350 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 11980
diff changeset
1004 {
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
1005 PyObject *str, *v;
12074
65f35b73e350 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 11980
diff changeset
1006 const char *errors = NULL;
65f35b73e350 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 11980
diff changeset
1007
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
1008 if (!PyArg_ParseTuple(args, "O|z:mbcs_encode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1009 &str, &errors))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1010 return NULL;
12074
65f35b73e350 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 11980
diff changeset
1011
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
1012 str = PyUnicode_FromObject(str);
73476
b3e532c1485e Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents: 73302
diff changeset
1013 if (str == NULL || PyUnicode_READY(str) < 0)
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1014 return NULL;
73476
b3e532c1485e Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents: 73302
diff changeset
1015 v = codec_tuple(PyUnicode_EncodeCodePage(CP_ACP, str, errors),
b3e532c1485e Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents: 73302
diff changeset
1016 PyUnicode_GET_LENGTH(str));
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
1017 Py_DECREF(str);
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
1018 return v;
12074
65f35b73e350 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 11980
diff changeset
1019 }
65f35b73e350 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 11980
diff changeset
1020
72976
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
1021 static PyObject *
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
1022 code_page_encode(PyObject *self,
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
1023 PyObject *args)
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
1024 {
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
1025 PyObject *str, *v;
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
1026 const char *errors = NULL;
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
1027 int code_page;
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
1028
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
1029 if (!PyArg_ParseTuple(args, "iO|z:code_page_encode",
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
1030 &code_page, &str, &errors))
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
1031 return NULL;
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
1032
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
1033 str = PyUnicode_FromObject(str);
73476
b3e532c1485e Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents: 73302
diff changeset
1034 if (str == NULL || PyUnicode_READY(str) < 0)
72976
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
1035 return NULL;
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
1036 v = codec_tuple(PyUnicode_EncodeCodePage(code_page,
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
1037 str,
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
1038 errors),
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
1039 PyUnicode_GET_LENGTH(str));
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
1040 Py_DECREF(str);
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
1041 return v;
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
1042 }
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
1043
71193
13e6d3cb2ecd Issue #9642: Uniformize the tests on the availability of the mbcs codec
Victor Stinner <victor.stinner@haypocalc.com>
parents: 68257
diff changeset
1044 #endif /* HAVE_MBCS */
12074
65f35b73e350 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 11980
diff changeset
1045
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24808
diff changeset
1046 /* --- Error handler registry --------------------------------------------- */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24808
diff changeset
1047
25974
040af9c736bb Add docstrings to register, lookup, register_error
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
1048 PyDoc_STRVAR(register_error__doc__,
040af9c736bb Add docstrings to register, lookup, register_error
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
1049 "register_error(errors, handler)\n\
040af9c736bb Add docstrings to register, lookup, register_error
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
1050 \n\
040af9c736bb Add docstrings to register, lookup, register_error
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
1051 Register the specified error handler under the name\n\
040af9c736bb Add docstrings to register, lookup, register_error
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
1052 errors. handler must be a callable object, that\n\
040af9c736bb Add docstrings to register, lookup, register_error
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
1053 will be called with an exception instance containing\n\
040af9c736bb Add docstrings to register, lookup, register_error
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
1054 information about the location of the encoding/decoding\n\
040af9c736bb Add docstrings to register, lookup, register_error
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
1055 error and must return a (replacement, new position) tuple.");
040af9c736bb Add docstrings to register, lookup, register_error
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
1056
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24808
diff changeset
1057 static PyObject *register_error(PyObject *self, PyObject *args)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24808
diff changeset
1058 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24808
diff changeset
1059 const char *name;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24808
diff changeset
1060 PyObject *handler;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24808
diff changeset
1061
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24808
diff changeset
1062 if (!PyArg_ParseTuple(args, "sO:register_error",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1063 &name, &handler))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1064 return NULL;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24808
diff changeset
1065 if (PyCodec_RegisterError(name, handler))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24808
diff changeset
1066 return NULL;
38669
19fd6141940d Partially merge trunk into p3yk. The removal of Mac/Tools is confusing svn
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
1067 Py_RETURN_NONE;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24808
diff changeset
1068 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24808
diff changeset
1069
25974
040af9c736bb Add docstrings to register, lookup, register_error
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
1070 PyDoc_STRVAR(lookup_error__doc__,
040af9c736bb Add docstrings to register, lookup, register_error
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
1071 "lookup_error(errors) -> handler\n\
040af9c736bb Add docstrings to register, lookup, register_error
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
1072 \n\
040af9c736bb Add docstrings to register, lookup, register_error
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
1073 Return the error handler for the specified error handling name\n\
040af9c736bb Add docstrings to register, lookup, register_error
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
1074 or raise a LookupError, if no handler exists under this name.");
040af9c736bb Add docstrings to register, lookup, register_error
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
1075
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24808
diff changeset
1076 static PyObject *lookup_error(PyObject *self, PyObject *args)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24808
diff changeset
1077 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24808
diff changeset
1078 const char *name;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24808
diff changeset
1079
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24808
diff changeset
1080 if (!PyArg_ParseTuple(args, "s:lookup_error",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1081 &name))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1082 return NULL;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24808
diff changeset
1083 return PyCodec_LookupError(name);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24808
diff changeset
1084 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24808
diff changeset
1085
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
1086 /* --- Module API --------------------------------------------------------- */
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
1087
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
1088 static PyMethodDef _codecs_functions[] = {
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1089 {"register", codec_register, METH_O,
25974
040af9c736bb Add docstrings to register, lookup, register_error
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
1090 register__doc__},
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1091 {"lookup", codec_lookup, METH_VARARGS,
25974
040af9c736bb Add docstrings to register, lookup, register_error
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
1092 lookup__doc__},
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1093 {"encode", codec_encode, METH_VARARGS,
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1094 encode__doc__},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1095 {"decode", codec_decode, METH_VARARGS,
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1096 decode__doc__},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1097 {"escape_encode", escape_encode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1098 {"escape_decode", escape_decode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1099 {"utf_8_encode", utf_8_encode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1100 {"utf_8_decode", utf_8_decode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1101 {"utf_7_encode", utf_7_encode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1102 {"utf_7_decode", utf_7_decode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1103 {"utf_16_encode", utf_16_encode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1104 {"utf_16_le_encode", utf_16_le_encode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1105 {"utf_16_be_encode", utf_16_be_encode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1106 {"utf_16_decode", utf_16_decode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1107 {"utf_16_le_decode", utf_16_le_decode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1108 {"utf_16_be_decode", utf_16_be_decode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1109 {"utf_16_ex_decode", utf_16_ex_decode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1110 {"utf_32_encode", utf_32_encode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1111 {"utf_32_le_encode", utf_32_le_encode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1112 {"utf_32_be_encode", utf_32_be_encode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1113 {"utf_32_decode", utf_32_decode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1114 {"utf_32_le_decode", utf_32_le_decode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1115 {"utf_32_be_decode", utf_32_be_decode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1116 {"utf_32_ex_decode", utf_32_ex_decode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1117 {"unicode_escape_encode", unicode_escape_encode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1118 {"unicode_escape_decode", unicode_escape_decode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1119 {"unicode_internal_encode", unicode_internal_encode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1120 {"unicode_internal_decode", unicode_internal_decode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1121 {"raw_unicode_escape_encode", raw_unicode_escape_encode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1122 {"raw_unicode_escape_decode", raw_unicode_escape_decode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1123 {"latin_1_encode", latin_1_encode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1124 {"latin_1_decode", latin_1_decode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1125 {"ascii_encode", ascii_encode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1126 {"ascii_decode", ascii_decode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1127 {"charmap_encode", charmap_encode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1128 {"charmap_decode", charmap_decode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1129 {"charmap_build", charmap_build, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1130 {"readbuffer_encode", readbuffer_encode, METH_VARARGS},
71193
13e6d3cb2ecd Issue #9642: Uniformize the tests on the availability of the mbcs codec
Victor Stinner <victor.stinner@haypocalc.com>
parents: 68257
diff changeset
1131 #ifdef HAVE_MBCS
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1132 {"mbcs_encode", mbcs_encode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1133 {"mbcs_decode", mbcs_decode, METH_VARARGS},
72976
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
1134 {"code_page_encode", code_page_encode, METH_VARARGS},
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
1135 {"code_page_decode", code_page_decode, METH_VARARGS},
12074
65f35b73e350 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 11980
diff changeset
1136 #endif
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1137 {"register_error", register_error, METH_VARARGS,
25974
040af9c736bb Add docstrings to register, lookup, register_error
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
1138 register_error__doc__},
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1139 {"lookup_error", lookup_error, METH_VARARGS,
25974
040af9c736bb Add docstrings to register, lookup, register_error
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
1140 lookup_error__doc__},
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1141 {NULL, NULL} /* sentinel */
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
1142 };
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
1143
47580
94f64f11fdd0 Implement PEP 3121: new module initialization and finalization API.
Martin v. Löwis <martin@v.loewis.de>
parents: 47301
diff changeset
1144 static struct PyModuleDef codecsmodule = {
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1145 PyModuleDef_HEAD_INIT,
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1146 "_codecs",
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1147 NULL,
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1148 -1,
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1149 _codecs_functions,
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1150 NULL,
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1151 NULL,
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1152 NULL,
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1153 NULL
47580
94f64f11fdd0 Implement PEP 3121: new module initialization and finalization API.
Martin v. Löwis <martin@v.loewis.de>
parents: 47301
diff changeset
1154 };
94f64f11fdd0 Implement PEP 3121: new module initialization and finalization API.
Martin v. Löwis <martin@v.loewis.de>
parents: 47301
diff changeset
1155
24471
f66b3a3509c7 Excise DL_EXPORT/DL_IMPORT from Modules/*. Required adding a prototype
Mark Hammond <mhammond@skippinet.com.au>
parents: 24071
diff changeset
1156 PyMODINIT_FUNC
47580
94f64f11fdd0 Implement PEP 3121: new module initialization and finalization API.
Martin v. Löwis <martin@v.loewis.de>
parents: 47301
diff changeset
1157 PyInit__codecs(void)
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
1158 {
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1159 return PyModule_Create(&codecsmodule);
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
1160 }