annotate Modules/_codecsmodule.c @ 73323:dad02a080bbc

Improve byte-compilation in packaging to be independent of -O or -B. The code I fixed to comply with PEP 3147 still had one bug: When run under python -O, some paths for pyc files would be pyo, because I called imp.cache_from_source without explicit debug_override argument in some places, and under -O that would return .pyo (this is well explained in the imp docs). Now all code (util.byte_compile, build_py, install_lib) can create .pyo files according to options given by users, without interference from the calling Python’s own optimize mode. On a related topic, I also removed the code that prevented byte compilation under python -B. The rationale is that packaging gives control over the creation of pyc files to the user with its own explicit option, and the behavior should not be changed if the calling Python happens to run with -B for whatever reason. I will argue that this is a bug fix and ask to be allowed to backport this change to distutils. Finally, I moved one nugget of information about the --compile and --optimize options from the source into the doc. It clears up a misunderstanding that I (and maybe other people) had.
author Éric Araujo <merwok@netwok.org>
date Thu, 03 Nov 2011 05:08:28 +0100
parents af0800b986b7
children 295fdfd4f422
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
1 /* ------------------------------------------------------------------------
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
2
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
3 _codecs -- Provides access to the codec registry and the builtin
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
4 codecs.
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
5
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
6 This module should never be imported directly. The standard library
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
7 module "codecs" wraps this builtin module for use within Python.
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
8
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
9 The codec registry is accessible via:
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
10
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
11 register(search_function) -> None
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
12
42324
4bbbe0c55fa2 Merged revisions 56443-56466 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42124
diff changeset
13 lookup(encoding) -> CodecInfo object
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
14
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
15 The builtin Unicode codecs use the following interface:
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
16
35825
cc17a582518a Whitespace cleanup.
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
17 <encoding>_encode(Unicode_object[,errors='strict']) ->
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
18 (string object, bytes consumed)
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
19
35825
cc17a582518a Whitespace cleanup.
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
20 <encoding>_decode(char_buffer_obj[,errors='strict']) ->
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
21 (Unicode object, bytes consumed)
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
22
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
23 <encoding>_encode() interfaces also accept non-Unicode object as
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
24 input. The objects are then converted to Unicode using
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
25 PyUnicode_FromObject() prior to applying the conversion.
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
26
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
27 These <encoding>s are available: utf_8, unicode_escape,
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
28 raw_unicode_escape, unicode_internal, latin_1, ascii (7-bit),
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
29 mbcs (on win32).
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
30
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
31
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
32 Written by Marc-Andre Lemburg (mal@lemburg.com).
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
33
13828
254f8b7c4a83 Changing the CNRI copyright notice according to CNRI's instructions.
Guido van Rossum <guido@python.org>
parents: 13651
diff changeset
34 Copyright (c) Corporation for National Research Initiatives.
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
35
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
36 ------------------------------------------------------------------------ */
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
37
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 35825
diff changeset
38 #define PY_SSIZE_T_CLEAN
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
39 #include "Python.h"
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
40
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
41 /* --- Registry ----------------------------------------------------------- */
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
42
25974
040af9c736bb Add docstrings to register, lookup, register_error
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
43 PyDoc_STRVAR(register__doc__,
040af9c736bb Add docstrings to register, lookup, register_error
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
44 "register(search_function)\n\
040af9c736bb Add docstrings to register, lookup, register_error
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
45 \n\
040af9c736bb Add docstrings to register, lookup, register_error
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
46 Register a codec search function. Search functions are expected to take\n\
040af9c736bb Add docstrings to register, lookup, register_error
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
47 one argument, the encoding name in all lower case letters, and return\n\
42324
4bbbe0c55fa2 Merged revisions 56443-56466 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42124
diff changeset
48 a tuple of functions (encoder, decoder, stream_reader, stream_writer)\n\
4bbbe0c55fa2 Merged revisions 56443-56466 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42124
diff changeset
49 (or a CodecInfo object).");
25974
040af9c736bb Add docstrings to register, lookup, register_error
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
50
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
51 static
38669
19fd6141940d Partially merge trunk into p3yk. The removal of Mac/Tools is confusing svn
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
52 PyObject *codec_register(PyObject *self, PyObject *search_function)
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
53 {
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
54 if (PyCodec_Register(search_function))
38669
19fd6141940d Partially merge trunk into p3yk. The removal of Mac/Tools is confusing svn
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
55 return NULL;
35825
cc17a582518a Whitespace cleanup.
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
56
38669
19fd6141940d Partially merge trunk into p3yk. The removal of Mac/Tools is confusing svn
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
57 Py_RETURN_NONE;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
58 }
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
59
25974
040af9c736bb Add docstrings to register, lookup, register_error
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
60 PyDoc_STRVAR(lookup__doc__,
42324
4bbbe0c55fa2 Merged revisions 56443-56466 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42124
diff changeset
61 "lookup(encoding) -> CodecInfo\n\
25974
040af9c736bb Add docstrings to register, lookup, register_error
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
62 \n\
040af9c736bb Add docstrings to register, lookup, register_error
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
63 Looks up a codec tuple in the Python codec registry and returns\n\
52260
c6688e9156b5 Merged revisions 70507-70508 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 50975
diff changeset
64 a CodecInfo object.");
25974
040af9c736bb Add docstrings to register, lookup, register_error
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
65
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
66 static
32572
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
67 PyObject *codec_lookup(PyObject *self, PyObject *args)
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
68 {
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
69 char *encoding;
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
70
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
71 if (!PyArg_ParseTuple(args, "s:lookup", &encoding))
38669
19fd6141940d Partially merge trunk into p3yk. The removal of Mac/Tools is confusing svn
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
72 return NULL;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
73
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
74 return _PyCodec_Lookup(encoding);
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
75 }
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
76
32572
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
77 PyDoc_STRVAR(encode__doc__,
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
78 "encode(obj, [encoding[,errors]]) -> object\n\
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
79 \n\
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
80 Encodes obj using the codec registered for encoding. encoding defaults\n\
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
81 to the default encoding. errors may be given to set a different error\n\
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
82 handling scheme. Default is 'strict' meaning that encoding errors raise\n\
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
83 a ValueError. Other possible values are 'ignore', 'replace' and\n\
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
84 'xmlcharrefreplace' as well as any other name registered with\n\
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
85 codecs.register_error that can handle ValueErrors.");
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
86
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
87 static PyObject *
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
88 codec_encode(PyObject *self, PyObject *args)
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
89 {
32602
ab979ff8b02a Change some declarations from ``char *`` to ``const char *``. Also added
Brett Cannon <bcannon@gmail.com>
parents: 32572
diff changeset
90 const char *encoding = NULL;
ab979ff8b02a Change some declarations from ``char *`` to ``const char *``. Also added
Brett Cannon <bcannon@gmail.com>
parents: 32572
diff changeset
91 const char *errors = NULL;
32572
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
92 PyObject *v;
35825
cc17a582518a Whitespace cleanup.
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
93
32572
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
94 if (!PyArg_ParseTuple(args, "O|ss:encode", &v, &encoding, &errors))
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
95 return NULL;
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
96
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
97 if (encoding == NULL)
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
98 encoding = PyUnicode_GetDefaultEncoding();
32572
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
99
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
100 /* Encode via the codec registry */
38669
19fd6141940d Partially merge trunk into p3yk. The removal of Mac/Tools is confusing svn
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
101 return PyCodec_Encode(v, encoding, errors);
32572
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
102 }
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
103
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
104 PyDoc_STRVAR(decode__doc__,
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
105 "decode(obj, [encoding[,errors]]) -> object\n\
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
106 \n\
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
107 Decodes obj using the codec registered for encoding. encoding defaults\n\
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
108 to the default encoding. errors may be given to set a different error\n\
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
109 handling scheme. Default is 'strict' meaning that encoding errors raise\n\
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
110 a ValueError. Other possible values are 'ignore' and 'replace'\n\
49493
bdf4da5d04ad merge from trunk
Benjamin Peterson <benjamin@python.org>
parents: 48683
diff changeset
111 as well as any other name registered with codecs.register_error that is\n\
32572
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
112 able to handle ValueErrors.");
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
113
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
114 static PyObject *
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
115 codec_decode(PyObject *self, PyObject *args)
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
116 {
32602
ab979ff8b02a Change some declarations from ``char *`` to ``const char *``. Also added
Brett Cannon <bcannon@gmail.com>
parents: 32572
diff changeset
117 const char *encoding = NULL;
ab979ff8b02a Change some declarations from ``char *`` to ``const char *``. Also added
Brett Cannon <bcannon@gmail.com>
parents: 32572
diff changeset
118 const char *errors = NULL;
32572
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
119 PyObject *v;
35825
cc17a582518a Whitespace cleanup.
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
120
32572
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
121 if (!PyArg_ParseTuple(args, "O|ss:decode", &v, &encoding, &errors))
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
122 return NULL;
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
123
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
124 if (encoding == NULL)
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
125 encoding = PyUnicode_GetDefaultEncoding();
32572
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
126
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
127 /* Decode via the codec registry */
38669
19fd6141940d Partially merge trunk into p3yk. The removal of Mac/Tools is confusing svn
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
128 return PyCodec_Decode(v, encoding, errors);
32572
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
129 }
8ea2cb1ec598 Add generic codecs.encode() and .decode() APIs that don't impose
Marc-André Lemburg <mal@egenix.com>
parents: 27680
diff changeset
130
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
131 /* --- Helpers ------------------------------------------------------------ */
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
132
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
133 static
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
134 PyObject *codec_tuple(PyObject *unicode,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
135 Py_ssize_t len)
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
136 {
38669
19fd6141940d Partially merge trunk into p3yk. The removal of Mac/Tools is confusing svn
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
137 PyObject *v;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
138 if (unicode == NULL)
38669
19fd6141940d Partially merge trunk into p3yk. The removal of Mac/Tools is confusing svn
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
139 return NULL;
19fd6141940d Partially merge trunk into p3yk. The removal of Mac/Tools is confusing svn
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
140 v = Py_BuildValue("On", unicode, len);
19fd6141940d Partially merge trunk into p3yk. The removal of Mac/Tools is confusing svn
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
141 Py_DECREF(unicode);
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
142 return v;
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
143 }
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
144
24808
ab63af3ec349 Patch #505705: Remove eval in pickle and cPickle.
Martin v. Löwis <martin@v.loewis.de>
parents: 24471
diff changeset
145 /* --- String codecs ------------------------------------------------------ */
ab63af3ec349 Patch #505705: Remove eval in pickle and cPickle.
Martin v. Löwis <martin@v.loewis.de>
parents: 24471
diff changeset
146 static PyObject *
ab63af3ec349 Patch #505705: Remove eval in pickle and cPickle.
Martin v. Löwis <martin@v.loewis.de>
parents: 24471
diff changeset
147 escape_decode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
148 PyObject *args)
24808
ab63af3ec349 Patch #505705: Remove eval in pickle and cPickle.
Martin v. Löwis <martin@v.loewis.de>
parents: 24471
diff changeset
149 {
ab63af3ec349 Patch #505705: Remove eval in pickle and cPickle.
Martin v. Löwis <martin@v.loewis.de>
parents: 24471
diff changeset
150 const char *errors = NULL;
ab63af3ec349 Patch #505705: Remove eval in pickle and cPickle.
Martin v. Löwis <martin@v.loewis.de>
parents: 24471
diff changeset
151 const char *data;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 35825
diff changeset
152 Py_ssize_t size;
35825
cc17a582518a Whitespace cleanup.
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
153
24808
ab63af3ec349 Patch #505705: Remove eval in pickle and cPickle.
Martin v. Löwis <martin@v.loewis.de>
parents: 24471
diff changeset
154 if (!PyArg_ParseTuple(args, "s#|z:escape_decode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
155 &data, &size, &errors))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
156 return NULL;
47301
23ab586c427a Renamed PyString to PyBytes
Christian Heimes <christian@cheimes.de>
parents: 43933
diff changeset
157 return codec_tuple(PyBytes_DecodeEscape(data, size, errors, 0, NULL),
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
158 size);
24808
ab63af3ec349 Patch #505705: Remove eval in pickle and cPickle.
Martin v. Löwis <martin@v.loewis.de>
parents: 24471
diff changeset
159 }
ab63af3ec349 Patch #505705: Remove eval in pickle and cPickle.
Martin v. Löwis <martin@v.loewis.de>
parents: 24471
diff changeset
160
ab63af3ec349 Patch #505705: Remove eval in pickle and cPickle.
Martin v. Löwis <martin@v.loewis.de>
parents: 24471
diff changeset
161 static PyObject *
ab63af3ec349 Patch #505705: Remove eval in pickle and cPickle.
Martin v. Löwis <martin@v.loewis.de>
parents: 24471
diff changeset
162 escape_encode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
163 PyObject *args)
24808
ab63af3ec349 Patch #505705: Remove eval in pickle and cPickle.
Martin v. Löwis <martin@v.loewis.de>
parents: 24471
diff changeset
164 {
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
165 PyObject *str;
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
166 Py_ssize_t size;
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
167 Py_ssize_t newsize;
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
168 const char *errors = NULL;
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
169 PyObject *v;
24808
ab63af3ec349 Patch #505705: Remove eval in pickle and cPickle.
Martin v. Löwis <martin@v.loewis.de>
parents: 24471
diff changeset
170
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
171 if (!PyArg_ParseTuple(args, "O!|z:escape_encode",
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
172 &PyBytes_Type, &str, &errors))
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
173 return NULL;
24808
ab63af3ec349 Patch #505705: Remove eval in pickle and cPickle.
Martin v. Löwis <martin@v.loewis.de>
parents: 24471
diff changeset
174
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
175 size = PyBytes_GET_SIZE(str);
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
176 newsize = 4*size;
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
177 if (newsize > PY_SSIZE_T_MAX || newsize / 4 != size) {
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
178 PyErr_SetString(PyExc_OverflowError,
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
179 "string is too large to encode");
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
180 return NULL;
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
181 }
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
182 v = PyBytes_FromStringAndSize(NULL, newsize);
41769
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41587
diff changeset
183
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
184 if (v == NULL) {
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
185 return NULL;
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
186 }
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
187 else {
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
188 register Py_ssize_t i;
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
189 register char c;
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
190 register char *p = PyBytes_AS_STRING(v);
24808
ab63af3ec349 Patch #505705: Remove eval in pickle and cPickle.
Martin v. Löwis <martin@v.loewis.de>
parents: 24471
diff changeset
191
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
192 for (i = 0; i < size; i++) {
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
193 /* There's at least enough room for a hex escape */
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
194 assert(newsize - (p - PyBytes_AS_STRING(v)) >= 4);
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
195 c = PyBytes_AS_STRING(str)[i];
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
196 if (c == '\'' || c == '\\')
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
197 *p++ = '\\', *p++ = c;
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
198 else if (c == '\t')
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
199 *p++ = '\\', *p++ = 't';
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
200 else if (c == '\n')
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
201 *p++ = '\\', *p++ = 'n';
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
202 else if (c == '\r')
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
203 *p++ = '\\', *p++ = 'r';
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
204 else if (c < ' ' || c >= 0x7f) {
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
205 *p++ = '\\';
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
206 *p++ = 'x';
72918
d76338eacf7c Issue #13088: Add shared Py_hexdigits constant to format a number into base 16
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72475
diff changeset
207 *p++ = Py_hexdigits[(c & 0xf0) >> 4];
d76338eacf7c Issue #13088: Add shared Py_hexdigits constant to format a number into base 16
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72475
diff changeset
208 *p++ = Py_hexdigits[c & 0xf];
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
209 }
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
210 else
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
211 *p++ = c;
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
212 }
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
213 *p = '\0';
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
214 if (_PyBytes_Resize(&v, (p - PyBytes_AS_STRING(v)))) {
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
215 return NULL;
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
216 }
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
217 }
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
218
62215
f33e464a9a60 Merged revisions 79780 via svnmerge from
Philip Jenvey <pjenvey@underboss.org>
parents: 62211
diff changeset
219 return codec_tuple(v, size);
24808
ab63af3ec349 Patch #505705: Remove eval in pickle and cPickle.
Martin v. Löwis <martin@v.loewis.de>
parents: 24471
diff changeset
220 }
ab63af3ec349 Patch #505705: Remove eval in pickle and cPickle.
Martin v. Löwis <martin@v.loewis.de>
parents: 24471
diff changeset
221
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
222 /* --- Decoder ------------------------------------------------------------ */
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
223
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
224 static PyObject *
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
225 unicode_internal_decode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
226 PyObject *args)
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
227 {
14733
814f421e0326 Added a true unicode_internal_encode function and fixed the
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
228 PyObject *obj;
814f421e0326 Added a true unicode_internal_encode function and fixed the
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
229 const char *errors = NULL;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
230 const char *data;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 35825
diff changeset
231 Py_ssize_t size;
35825
cc17a582518a Whitespace cleanup.
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
232
14733
814f421e0326 Added a true unicode_internal_encode function and fixed the
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
233 if (!PyArg_ParseTuple(args, "O|z:unicode_internal_decode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
234 &obj, &errors))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
235 return NULL;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
236
27680
b19aaaeb6d64 Fix for [ 543344 ] Interpreter crashes when recoding; suggested
Marc-André Lemburg <mal@egenix.com>
parents: 25974
diff changeset
237 if (PyUnicode_Check(obj)) {
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
238 Py_INCREF(obj);
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
239 return codec_tuple(obj, PyUnicode_GET_SIZE(obj));
27680
b19aaaeb6d64 Fix for [ 543344 ] Interpreter crashes when recoding; suggested
Marc-André Lemburg <mal@egenix.com>
parents: 25974
diff changeset
240 }
14733
814f421e0326 Added a true unicode_internal_encode function and fixed the
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
241 else {
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
242 if (PyObject_AsReadBuffer(obj, (const void **)&data, &size))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
243 return NULL;
35485
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34707
diff changeset
244
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
245 return codec_tuple(_PyUnicode_DecodeUnicodeInternal(data, size, errors),
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
246 size);
14733
814f421e0326 Added a true unicode_internal_encode function and fixed the
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
247 }
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
248 }
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
249
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
250 static PyObject *
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19343
diff changeset
251 utf_7_decode(PyObject *self,
43933
9c4bf18e5d30 Merged revisions 59056-59076 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents: 43777
diff changeset
252 PyObject *args)
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19343
diff changeset
253 {
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
254 Py_buffer pbuf;
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19343
diff changeset
255 const char *errors = NULL;
43933
9c4bf18e5d30 Merged revisions 59056-59076 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents: 43777
diff changeset
256 int final = 0;
9c4bf18e5d30 Merged revisions 59056-59076 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents: 43777
diff changeset
257 Py_ssize_t consumed;
9c4bf18e5d30 Merged revisions 59056-59076 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents: 43777
diff changeset
258 PyObject *decoded = NULL;
35825
cc17a582518a Whitespace cleanup.
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
259
50975
ec6f29608bf0 Merged revisions 68855 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 49493
diff changeset
260 if (!PyArg_ParseTuple(args, "y*|zi:utf_7_decode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
261 &pbuf, &errors, &final))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
262 return NULL;
48683
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
263 consumed = pbuf.len;
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19343
diff changeset
264
48683
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
265 decoded = PyUnicode_DecodeUTF7Stateful(pbuf.buf, pbuf.len, errors,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
266 final ? NULL : &consumed);
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
267 PyBuffer_Release(&pbuf);
43933
9c4bf18e5d30 Merged revisions 59056-59076 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents: 43777
diff changeset
268 if (decoded == NULL)
9c4bf18e5d30 Merged revisions 59056-59076 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents: 43777
diff changeset
269 return NULL;
9c4bf18e5d30 Merged revisions 59056-59076 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents: 43777
diff changeset
270 return codec_tuple(decoded, consumed);
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19343
diff changeset
271 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19343
diff changeset
272
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19343
diff changeset
273 static PyObject *
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
274 utf_8_decode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
275 PyObject *args)
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
276 {
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
277 Py_buffer pbuf;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
278 const char *errors = NULL;
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 32602
diff changeset
279 int final = 0;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 35825
diff changeset
280 Py_ssize_t consumed;
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 32602
diff changeset
281 PyObject *decoded = NULL;
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 32602
diff changeset
282
50975
ec6f29608bf0 Merged revisions 68855 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 49493
diff changeset
283 if (!PyArg_ParseTuple(args, "y*|zi:utf_8_decode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
284 &pbuf, &errors, &final))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
285 return NULL;
48683
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
286 consumed = pbuf.len;
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
287
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
288 decoded = PyUnicode_DecodeUTF8Stateful(pbuf.buf, pbuf.len, errors,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
289 final ? NULL : &consumed);
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
290 PyBuffer_Release(&pbuf);
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 32602
diff changeset
291 if (decoded == NULL)
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
292 return NULL;
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 32602
diff changeset
293 return codec_tuple(decoded, consumed);
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
294 }
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
295
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
296 static PyObject *
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
297 utf_16_decode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
298 PyObject *args)
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
299 {
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
300 Py_buffer pbuf;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
301 const char *errors = NULL;
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
302 int byteorder = 0;
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 32602
diff changeset
303 int final = 0;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 35825
diff changeset
304 Py_ssize_t consumed;
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 32602
diff changeset
305 PyObject *decoded;
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 32602
diff changeset
306
50975
ec6f29608bf0 Merged revisions 68855 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 49493
diff changeset
307 if (!PyArg_ParseTuple(args, "y*|zi:utf_16_decode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
308 &pbuf, &errors, &final))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
309 return NULL;
48683
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
310 consumed = pbuf.len; /* This is overwritten unless final is true. */
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
311 decoded = PyUnicode_DecodeUTF16Stateful(pbuf.buf, pbuf.len, errors,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
312 &byteorder, final ? NULL : &consumed);
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
313 PyBuffer_Release(&pbuf);
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 32602
diff changeset
314 if (decoded == NULL)
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
315 return NULL;
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 32602
diff changeset
316 return codec_tuple(decoded, consumed);
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
317 }
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
318
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
319 static PyObject *
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
320 utf_16_le_decode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
321 PyObject *args)
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
322 {
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
323 Py_buffer pbuf;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
324 const char *errors = NULL;
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
325 int byteorder = -1;
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 32602
diff changeset
326 int final = 0;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 35825
diff changeset
327 Py_ssize_t consumed;
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 32602
diff changeset
328 PyObject *decoded = NULL;
35825
cc17a582518a Whitespace cleanup.
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
329
50975
ec6f29608bf0 Merged revisions 68855 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 49493
diff changeset
330 if (!PyArg_ParseTuple(args, "y*|zi:utf_16_le_decode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
331 &pbuf, &errors, &final))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
332 return NULL;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 35825
diff changeset
333
48683
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
334 consumed = pbuf.len; /* This is overwritten unless final is true. */
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
335 decoded = PyUnicode_DecodeUTF16Stateful(pbuf.buf, pbuf.len, errors,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
336 &byteorder, final ? NULL : &consumed);
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
337 PyBuffer_Release(&pbuf);
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 32602
diff changeset
338 if (decoded == NULL)
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
339 return NULL;
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 32602
diff changeset
340 return codec_tuple(decoded, consumed);
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
341 }
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
342
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
343 static PyObject *
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
344 utf_16_be_decode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
345 PyObject *args)
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
346 {
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
347 Py_buffer pbuf;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
348 const char *errors = NULL;
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
349 int byteorder = 1;
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 32602
diff changeset
350 int final = 0;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 35825
diff changeset
351 Py_ssize_t consumed;
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 32602
diff changeset
352 PyObject *decoded = NULL;
35825
cc17a582518a Whitespace cleanup.
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
353
50975
ec6f29608bf0 Merged revisions 68855 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 49493
diff changeset
354 if (!PyArg_ParseTuple(args, "y*|zi:utf_16_be_decode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
355 &pbuf, &errors, &final))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
356 return NULL;
48683
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
357
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
358 consumed = pbuf.len; /* This is overwritten unless final is true. */
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
359 decoded = PyUnicode_DecodeUTF16Stateful(pbuf.buf, pbuf.len, errors,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
360 &byteorder, final ? NULL : &consumed);
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
361 PyBuffer_Release(&pbuf);
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 32602
diff changeset
362 if (decoded == NULL)
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
363 return NULL;
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 32602
diff changeset
364 return codec_tuple(decoded, consumed);
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
365 }
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
366
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
367 /* This non-standard version also provides access to the byteorder
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
368 parameter of the builtin UTF-16 codec.
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
369
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
370 It returns a tuple (unicode, bytesread, byteorder) with byteorder
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
371 being the value in effect at the end of data.
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
372
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
373 */
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
374
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
375 static PyObject *
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
376 utf_16_ex_decode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
377 PyObject *args)
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
378 {
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
379 Py_buffer pbuf;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
380 const char *errors = NULL;
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
381 int byteorder = 0;
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
382 PyObject *unicode, *tuple;
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 32602
diff changeset
383 int final = 0;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 35825
diff changeset
384 Py_ssize_t consumed;
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 32602
diff changeset
385
50975
ec6f29608bf0 Merged revisions 68855 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 49493
diff changeset
386 if (!PyArg_ParseTuple(args, "y*|zii:utf_16_ex_decode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
387 &pbuf, &errors, &byteorder, &final))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
388 return NULL;
48683
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
389 consumed = pbuf.len; /* This is overwritten unless final is true. */
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
390 unicode = PyUnicode_DecodeUTF16Stateful(pbuf.buf, pbuf.len, errors,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
391 &byteorder, final ? NULL : &consumed);
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
392 PyBuffer_Release(&pbuf);
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
393 if (unicode == NULL)
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
394 return NULL;
38669
19fd6141940d Partially merge trunk into p3yk. The removal of Mac/Tools is confusing svn
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
395 tuple = Py_BuildValue("Oni", unicode, consumed, byteorder);
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
396 Py_DECREF(unicode);
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
397 return tuple;
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
398 }
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
399
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
400 static PyObject *
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
401 utf_32_decode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
402 PyObject *args)
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
403 {
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
404 Py_buffer pbuf;
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
405 const char *errors = NULL;
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
406 int byteorder = 0;
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
407 int final = 0;
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
408 Py_ssize_t consumed;
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
409 PyObject *decoded;
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
410
50975
ec6f29608bf0 Merged revisions 68855 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 49493
diff changeset
411 if (!PyArg_ParseTuple(args, "y*|zi:utf_32_decode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
412 &pbuf, &errors, &final))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
413 return NULL;
48683
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
414 consumed = pbuf.len; /* This is overwritten unless final is true. */
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
415 decoded = PyUnicode_DecodeUTF32Stateful(pbuf.buf, pbuf.len, errors,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
416 &byteorder, final ? NULL : &consumed);
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
417 PyBuffer_Release(&pbuf);
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
418 if (decoded == NULL)
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
419 return NULL;
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
420 return codec_tuple(decoded, consumed);
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
421 }
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
422
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
423 static PyObject *
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
424 utf_32_le_decode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
425 PyObject *args)
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
426 {
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
427 Py_buffer pbuf;
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
428 const char *errors = NULL;
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
429 int byteorder = -1;
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
430 int final = 0;
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
431 Py_ssize_t consumed;
48683
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
432 PyObject *decoded;
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
433
50975
ec6f29608bf0 Merged revisions 68855 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 49493
diff changeset
434 if (!PyArg_ParseTuple(args, "y*|zi:utf_32_le_decode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
435 &pbuf, &errors, &final))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
436 return NULL;
48683
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
437 consumed = pbuf.len; /* This is overwritten unless final is true. */
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
438 decoded = PyUnicode_DecodeUTF32Stateful(pbuf.buf, pbuf.len, errors,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
439 &byteorder, final ? NULL : &consumed);
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
440 PyBuffer_Release(&pbuf);
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
441 if (decoded == NULL)
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
442 return NULL;
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
443 return codec_tuple(decoded, consumed);
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
444 }
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
445
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
446 static PyObject *
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
447 utf_32_be_decode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
448 PyObject *args)
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
449 {
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
450 Py_buffer pbuf;
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
451 const char *errors = NULL;
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
452 int byteorder = 1;
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
453 int final = 0;
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
454 Py_ssize_t consumed;
48683
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
455 PyObject *decoded;
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
456
50975
ec6f29608bf0 Merged revisions 68855 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 49493
diff changeset
457 if (!PyArg_ParseTuple(args, "y*|zi:utf_32_be_decode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
458 &pbuf, &errors, &final))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
459 return NULL;
48683
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
460 consumed = pbuf.len; /* This is overwritten unless final is true. */
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
461 decoded = PyUnicode_DecodeUTF32Stateful(pbuf.buf, pbuf.len, errors,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
462 &byteorder, final ? NULL : &consumed);
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
463 PyBuffer_Release(&pbuf);
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
464 if (decoded == NULL)
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
465 return NULL;
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
466 return codec_tuple(decoded, consumed);
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
467 }
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
468
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
469 /* This non-standard version also provides access to the byteorder
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
470 parameter of the builtin UTF-32 codec.
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
471
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
472 It returns a tuple (unicode, bytesread, byteorder) with byteorder
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
473 being the value in effect at the end of data.
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
474
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
475 */
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
476
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
477 static PyObject *
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
478 utf_32_ex_decode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
479 PyObject *args)
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
480 {
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
481 Py_buffer pbuf;
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
482 const char *errors = NULL;
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
483 int byteorder = 0;
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
484 PyObject *unicode, *tuple;
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
485 int final = 0;
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
486 Py_ssize_t consumed;
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
487
50975
ec6f29608bf0 Merged revisions 68855 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 49493
diff changeset
488 if (!PyArg_ParseTuple(args, "y*|zii:utf_32_ex_decode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
489 &pbuf, &errors, &byteorder, &final))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
490 return NULL;
48683
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
491 consumed = pbuf.len; /* This is overwritten unless final is true. */
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
492 unicode = PyUnicode_DecodeUTF32Stateful(pbuf.buf, pbuf.len, errors,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
493 &byteorder, final ? NULL : &consumed);
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
494 PyBuffer_Release(&pbuf);
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
495 if (unicode == NULL)
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
496 return NULL;
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
497 tuple = Py_BuildValue("Oni", unicode, consumed, byteorder);
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
498 Py_DECREF(unicode);
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
499 return tuple;
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
500 }
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
501
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
502 static PyObject *
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
503 unicode_escape_decode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
504 PyObject *args)
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
505 {
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
506 Py_buffer pbuf;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
507 const char *errors = NULL;
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
508 PyObject *unicode;
35825
cc17a582518a Whitespace cleanup.
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
509
48683
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
510 if (!PyArg_ParseTuple(args, "s*|z:unicode_escape_decode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
511 &pbuf, &errors))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
512 return NULL;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
513
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
514 unicode = PyUnicode_DecodeUnicodeEscape(pbuf.buf, pbuf.len, errors);
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
515 PyBuffer_Release(&pbuf);
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
516 return codec_tuple(unicode, pbuf.len);
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
517 }
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
518
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
519 static PyObject *
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
520 raw_unicode_escape_decode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
521 PyObject *args)
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
522 {
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
523 Py_buffer pbuf;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
524 const char *errors = NULL;
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
525 PyObject *unicode;
35825
cc17a582518a Whitespace cleanup.
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
526
48683
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
527 if (!PyArg_ParseTuple(args, "s*|z:raw_unicode_escape_decode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
528 &pbuf, &errors))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
529 return NULL;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
530
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
531 unicode = PyUnicode_DecodeRawUnicodeEscape(pbuf.buf, pbuf.len, errors);
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
532 PyBuffer_Release(&pbuf);
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
533 return codec_tuple(unicode, pbuf.len);
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
534 }
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
535
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
536 static PyObject *
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
537 latin_1_decode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
538 PyObject *args)
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
539 {
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
540 Py_buffer pbuf;
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
541 PyObject *unicode;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
542 const char *errors = NULL;
35825
cc17a582518a Whitespace cleanup.
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
543
50975
ec6f29608bf0 Merged revisions 68855 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 49493
diff changeset
544 if (!PyArg_ParseTuple(args, "y*|z:latin_1_decode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
545 &pbuf, &errors))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
546 return NULL;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
547
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
548 unicode = PyUnicode_DecodeLatin1(pbuf.buf, pbuf.len, errors);
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
549 PyBuffer_Release(&pbuf);
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
550 return codec_tuple(unicode, pbuf.len);
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
551 }
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
552
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
553 static PyObject *
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
554 ascii_decode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
555 PyObject *args)
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
556 {
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
557 Py_buffer pbuf;
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
558 PyObject *unicode;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
559 const char *errors = NULL;
35825
cc17a582518a Whitespace cleanup.
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
560
50975
ec6f29608bf0 Merged revisions 68855 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 49493
diff changeset
561 if (!PyArg_ParseTuple(args, "y*|z:ascii_decode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
562 &pbuf, &errors))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
563 return NULL;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
564
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
565 unicode = PyUnicode_DecodeASCII(pbuf.buf, pbuf.len, errors);
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
566 PyBuffer_Release(&pbuf);
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
567 return codec_tuple(unicode, pbuf.len);
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
568 }
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
569
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
570 static PyObject *
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
571 charmap_decode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
572 PyObject *args)
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
573 {
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
574 Py_buffer pbuf;
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
575 PyObject *unicode;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
576 const char *errors = NULL;
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
577 PyObject *mapping = NULL;
35825
cc17a582518a Whitespace cleanup.
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
578
50975
ec6f29608bf0 Merged revisions 68855 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 49493
diff changeset
579 if (!PyArg_ParseTuple(args, "y*|zO:charmap_decode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
580 &pbuf, &errors, &mapping))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
581 return NULL;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
582 if (mapping == Py_None)
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
583 mapping = NULL;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
584
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
585 unicode = PyUnicode_DecodeCharmap(pbuf.buf, pbuf.len, mapping, errors);
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
586 PyBuffer_Release(&pbuf);
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
587 return codec_tuple(unicode, pbuf.len);
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
588 }
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
589
71193
13e6d3cb2ecd Issue #9642: Uniformize the tests on the availability of the mbcs codec
Victor Stinner <victor.stinner@haypocalc.com>
parents: 68257
diff changeset
590 #ifdef HAVE_MBCS
12074
65f35b73e350 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 11980
diff changeset
591
65f35b73e350 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 11980
diff changeset
592 static PyObject *
65f35b73e350 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 11980
diff changeset
593 mbcs_decode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
594 PyObject *args)
12074
65f35b73e350 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 11980
diff changeset
595 {
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
596 Py_buffer pbuf;
12074
65f35b73e350 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 11980
diff changeset
597 const char *errors = NULL;
39442
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 38673
diff changeset
598 int final = 0;
48683
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
599 Py_ssize_t consumed;
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
600 PyObject *decoded = NULL;
12074
65f35b73e350 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 11980
diff changeset
601
50975
ec6f29608bf0 Merged revisions 68855 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 49493
diff changeset
602 if (!PyArg_ParseTuple(args, "y*|zi:mbcs_decode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
603 &pbuf, &errors, &final))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
604 return NULL;
48683
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
605 consumed = pbuf.len;
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
606
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
607 decoded = PyUnicode_DecodeMBCSStateful(pbuf.buf, pbuf.len, errors,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
608 final ? NULL : &consumed);
62211
3375239d4370 Merged revisions 81862 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 61467
diff changeset
609 PyBuffer_Release(&pbuf);
48683
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
610 if (decoded == NULL)
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
611 return NULL;
48683
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
612 return codec_tuple(decoded, consumed);
12074
65f35b73e350 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 11980
diff changeset
613 }
65f35b73e350 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 11980
diff changeset
614
72976
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
615 static PyObject *
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
616 code_page_decode(PyObject *self,
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
617 PyObject *args)
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
618 {
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
619 Py_buffer pbuf;
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
620 const char *errors = NULL;
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
621 int final = 0;
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
622 Py_ssize_t consumed;
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
623 PyObject *decoded = NULL;
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
624 int code_page;
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
625
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
626 if (!PyArg_ParseTuple(args, "iy*|zi:code_page_decode",
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
627 &code_page, &pbuf, &errors, &final))
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
628 return NULL;
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
629 consumed = pbuf.len;
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
630
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
631 decoded = PyUnicode_DecodeCodePageStateful(code_page,
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
632 pbuf.buf, pbuf.len, errors,
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
633 final ? NULL : &consumed);
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
634 PyBuffer_Release(&pbuf);
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
635 if (decoded == NULL)
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
636 return NULL;
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
637 return codec_tuple(decoded, consumed);
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
638 }
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
639
71193
13e6d3cb2ecd Issue #9642: Uniformize the tests on the availability of the mbcs codec
Victor Stinner <victor.stinner@haypocalc.com>
parents: 68257
diff changeset
640 #endif /* HAVE_MBCS */
12074
65f35b73e350 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 11980
diff changeset
641
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
642 /* --- Encoder ------------------------------------------------------------ */
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
643
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
644 static PyObject *
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
645 readbuffer_encode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
646 PyObject *args)
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
647 {
48683
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
648 Py_buffer pdata;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
649 const char *data;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 35825
diff changeset
650 Py_ssize_t size;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
651 const char *errors = NULL;
48683
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
652 PyObject *result;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
653
48683
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
654 if (!PyArg_ParseTuple(args, "s*|z:readbuffer_encode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
655 &pdata, &errors))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
656 return NULL;
48683
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
657 data = pdata.buf;
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
658 size = pdata.len;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
659
48683
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
660 result = PyBytes_FromStringAndSize(data, size);
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
661 PyBuffer_Release(&pdata);
cd3b0b7fd8e1 Merged revisions 65654 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 47580
diff changeset
662 return codec_tuple(result, size);
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
663 }
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
664
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
665 static PyObject *
14733
814f421e0326 Added a true unicode_internal_encode function and fixed the
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
666 unicode_internal_encode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
667 PyObject *args)
14733
814f421e0326 Added a true unicode_internal_encode function and fixed the
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
668 {
814f421e0326 Added a true unicode_internal_encode function and fixed the
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
669 PyObject *obj;
814f421e0326 Added a true unicode_internal_encode function and fixed the
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
670 const char *errors = NULL;
814f421e0326 Added a true unicode_internal_encode function and fixed the
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
671 const char *data;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 35825
diff changeset
672 Py_ssize_t size;
35825
cc17a582518a Whitespace cleanup.
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
673
14733
814f421e0326 Added a true unicode_internal_encode function and fixed the
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
674 if (!PyArg_ParseTuple(args, "O|z:unicode_internal_encode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
675 &obj, &errors))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
676 return NULL;
14733
814f421e0326 Added a true unicode_internal_encode function and fixed the
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
677
814f421e0326 Added a true unicode_internal_encode function and fixed the
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
678 if (PyUnicode_Check(obj)) {
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
679 data = PyUnicode_AS_DATA(obj);
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
680 size = PyUnicode_GET_DATA_SIZE(obj);
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
681 return codec_tuple(PyBytes_FromStringAndSize(data, size),
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
682 PyUnicode_GET_SIZE(obj));
14733
814f421e0326 Added a true unicode_internal_encode function and fixed the
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
683 }
814f421e0326 Added a true unicode_internal_encode function and fixed the
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
684 else {
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
685 if (PyObject_AsReadBuffer(obj, (const void **)&data, &size))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
686 return NULL;
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
687 return codec_tuple(PyBytes_FromStringAndSize(data, size), size);
14733
814f421e0326 Added a true unicode_internal_encode function and fixed the
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
688 }
814f421e0326 Added a true unicode_internal_encode function and fixed the
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
689 }
814f421e0326 Added a true unicode_internal_encode function and fixed the
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
690
814f421e0326 Added a true unicode_internal_encode function and fixed the
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
691 static PyObject *
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19343
diff changeset
692 utf_7_encode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
693 PyObject *args)
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19343
diff changeset
694 {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19343
diff changeset
695 PyObject *str, *v;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19343
diff changeset
696 const char *errors = NULL;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19343
diff changeset
697
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19343
diff changeset
698 if (!PyArg_ParseTuple(args, "O|z:utf_7_encode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
699 &str, &errors))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
700 return NULL;
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19343
diff changeset
701
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19343
diff changeset
702 str = PyUnicode_FromObject(str);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19343
diff changeset
703 if (str == NULL)
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
704 return NULL;
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19343
diff changeset
705 v = codec_tuple(PyUnicode_EncodeUTF7(PyUnicode_AS_UNICODE(str),
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
706 PyUnicode_GET_SIZE(str),
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
707 0,
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
708 0,
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
709 errors),
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
710 PyUnicode_GET_SIZE(str));
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19343
diff changeset
711 Py_DECREF(str);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19343
diff changeset
712 return v;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19343
diff changeset
713 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19343
diff changeset
714
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19343
diff changeset
715 static PyObject *
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
716 utf_8_encode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
717 PyObject *args)
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
718 {
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
719 PyObject *str, *v;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
720 const char *errors = NULL;
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
721
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
722 if (!PyArg_ParseTuple(args, "O|z:utf_8_encode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
723 &str, &errors))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
724 return NULL;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
725
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
726 str = PyUnicode_FromObject(str);
72475
8beaa9a37387 Implement PEP 393.
Martin v. Löwis <martin@v.loewis.de>
parents: 71193
diff changeset
727 if (str == NULL || PyUnicode_READY(str) == -1)
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
728 return NULL;
72475
8beaa9a37387 Implement PEP 393.
Martin v. Löwis <martin@v.loewis.de>
parents: 71193
diff changeset
729 v = codec_tuple(PyUnicode_AsEncodedString(str, "utf-8", errors),
8beaa9a37387 Implement PEP 393.
Martin v. Löwis <martin@v.loewis.de>
parents: 71193
diff changeset
730 PyUnicode_GET_LENGTH(str));
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
731 Py_DECREF(str);
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
732 return v;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
733 }
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
734
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
735 /* This version provides access to the byteorder parameter of the
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
736 builtin UTF-16 codecs as optional third argument. It defaults to 0
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
737 which means: use the native byte order and prepend the data with a
35825
cc17a582518a Whitespace cleanup.
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
738 BOM mark.
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
739
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
740 */
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
741
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
742 static PyObject *
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
743 utf_16_encode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
744 PyObject *args)
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
745 {
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
746 PyObject *str, *v;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
747 const char *errors = NULL;
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
748 int byteorder = 0;
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
749
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
750 if (!PyArg_ParseTuple(args, "O|zi:utf_16_encode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
751 &str, &errors, &byteorder))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
752 return NULL;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
753
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
754 str = PyUnicode_FromObject(str);
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
755 if (str == NULL)
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
756 return NULL;
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
757 v = codec_tuple(PyUnicode_EncodeUTF16(PyUnicode_AS_UNICODE(str),
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
758 PyUnicode_GET_SIZE(str),
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
759 errors,
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
760 byteorder),
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
761 PyUnicode_GET_SIZE(str));
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
762 Py_DECREF(str);
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
763 return v;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
764 }
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
765
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
766 static PyObject *
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
767 utf_16_le_encode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
768 PyObject *args)
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
769 {
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
770 PyObject *str, *v;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
771 const char *errors = NULL;
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
772
18139
354a4df09e38 Fix for bug [ #433047 ] missing args to PyArg_ParseTuple
Marc-André Lemburg <mal@egenix.com>
parents: 14733
diff changeset
773 if (!PyArg_ParseTuple(args, "O|z:utf_16_le_encode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
774 &str, &errors))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
775 return NULL;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
776
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
777 str = PyUnicode_FromObject(str);
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
778 if (str == NULL)
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
779 return NULL;
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
780 v = codec_tuple(PyUnicode_EncodeUTF16(PyUnicode_AS_UNICODE(str),
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
781 PyUnicode_GET_SIZE(str),
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
782 errors,
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
783 -1),
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
784 PyUnicode_GET_SIZE(str));
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
785 Py_DECREF(str);
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
786 return v;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
787 }
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
788
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
789 static PyObject *
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
790 utf_16_be_encode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
791 PyObject *args)
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
792 {
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
793 PyObject *str, *v;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
794 const char *errors = NULL;
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
795
18139
354a4df09e38 Fix for bug [ #433047 ] missing args to PyArg_ParseTuple
Marc-André Lemburg <mal@egenix.com>
parents: 14733
diff changeset
796 if (!PyArg_ParseTuple(args, "O|z:utf_16_be_encode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
797 &str, &errors))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
798 return NULL;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
799
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
800 str = PyUnicode_FromObject(str);
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
801 if (str == NULL)
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
802 return NULL;
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
803 v = codec_tuple(PyUnicode_EncodeUTF16(PyUnicode_AS_UNICODE(str),
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
804 PyUnicode_GET_SIZE(str),
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
805 errors,
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
806 +1),
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
807 PyUnicode_GET_SIZE(str));
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
808 Py_DECREF(str);
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
809 return v;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
810 }
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
811
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
812 /* This version provides access to the byteorder parameter of the
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
813 builtin UTF-32 codecs as optional third argument. It defaults to 0
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
814 which means: use the native byte order and prepend the data with a
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
815 BOM mark.
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
816
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
817 */
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
818
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
819 static PyObject *
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
820 utf_32_encode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
821 PyObject *args)
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
822 {
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
823 PyObject *str, *v;
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
824 const char *errors = NULL;
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
825 int byteorder = 0;
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
826
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
827 if (!PyArg_ParseTuple(args, "O|zi:utf_32_encode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
828 &str, &errors, &byteorder))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
829 return NULL;
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
830
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
831 str = PyUnicode_FromObject(str);
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
832 if (str == NULL)
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
833 return NULL;
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
834 v = codec_tuple(PyUnicode_EncodeUTF32(PyUnicode_AS_UNICODE(str),
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
835 PyUnicode_GET_SIZE(str),
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
836 errors,
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
837 byteorder),
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
838 PyUnicode_GET_SIZE(str));
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
839 Py_DECREF(str);
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
840 return v;
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
841 }
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
842
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
843 static PyObject *
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
844 utf_32_le_encode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
845 PyObject *args)
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
846 {
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
847 PyObject *str, *v;
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
848 const char *errors = NULL;
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
849
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
850 if (!PyArg_ParseTuple(args, "O|z:utf_32_le_encode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
851 &str, &errors))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
852 return NULL;
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
853
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
854 str = PyUnicode_FromObject(str);
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
855 if (str == NULL)
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
856 return NULL;
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
857 v = codec_tuple(PyUnicode_EncodeUTF32(PyUnicode_AS_UNICODE(str),
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
858 PyUnicode_GET_SIZE(str),
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
859 errors,
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
860 -1),
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
861 PyUnicode_GET_SIZE(str));
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
862 Py_DECREF(str);
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
863 return v;
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
864 }
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
865
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
866 static PyObject *
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
867 utf_32_be_encode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
868 PyObject *args)
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
869 {
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
870 PyObject *str, *v;
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
871 const char *errors = NULL;
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
872
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
873 if (!PyArg_ParseTuple(args, "O|z:utf_32_be_encode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
874 &str, &errors))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
875 return NULL;
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
876
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
877 str = PyUnicode_FromObject(str);
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
878 if (str == NULL)
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
879 return NULL;
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
880 v = codec_tuple(PyUnicode_EncodeUTF32(PyUnicode_AS_UNICODE(str),
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
881 PyUnicode_GET_SIZE(str),
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
882 errors,
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
883 +1),
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
884 PyUnicode_GET_SIZE(str));
42626
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
885 Py_DECREF(str);
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
886 return v;
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
887 }
f1a2d51f1cab Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents: 42324
diff changeset
888
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
889 static PyObject *
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
890 unicode_escape_encode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
891 PyObject *args)
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
892 {
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
893 PyObject *str, *v;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
894 const char *errors = NULL;
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
895
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
896 if (!PyArg_ParseTuple(args, "O|z:unicode_escape_encode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
897 &str, &errors))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
898 return NULL;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
899
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
900 str = PyUnicode_FromObject(str);
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
901 if (str == NULL)
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
902 return NULL;
35825
cc17a582518a Whitespace cleanup.
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
903 v = codec_tuple(PyUnicode_EncodeUnicodeEscape(PyUnicode_AS_UNICODE(str),
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
904 PyUnicode_GET_SIZE(str)),
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
905 PyUnicode_GET_SIZE(str));
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
906 Py_DECREF(str);
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
907 return v;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
908 }
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
909
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
910 static PyObject *
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
911 raw_unicode_escape_encode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
912 PyObject *args)
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
913 {
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
914 PyObject *str, *v;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
915 const char *errors = NULL;
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
916
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
917 if (!PyArg_ParseTuple(args, "O|z:raw_unicode_escape_encode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
918 &str, &errors))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
919 return NULL;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
920
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
921 str = PyUnicode_FromObject(str);
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
922 if (str == NULL)
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
923 return NULL;
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
924 v = codec_tuple(PyUnicode_EncodeRawUnicodeEscape(
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
925 PyUnicode_AS_UNICODE(str),
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
926 PyUnicode_GET_SIZE(str)),
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
927 PyUnicode_GET_SIZE(str));
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
928 Py_DECREF(str);
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
929 return v;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
930 }
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
931
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
932 static PyObject *
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
933 latin_1_encode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
934 PyObject *args)
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
935 {
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
936 PyObject *str, *v;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
937 const char *errors = NULL;
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
938
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
939 if (!PyArg_ParseTuple(args, "O|z:latin_1_encode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
940 &str, &errors))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
941 return NULL;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
942
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
943 str = PyUnicode_FromObject(str);
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
944 if (str == NULL)
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
945 return NULL;
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
946 v = codec_tuple(PyUnicode_EncodeLatin1(
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
947 PyUnicode_AS_UNICODE(str),
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
948 PyUnicode_GET_SIZE(str),
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
949 errors),
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
950 PyUnicode_GET_SIZE(str));
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
951 Py_DECREF(str);
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
952 return v;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
953 }
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
954
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
955 static PyObject *
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
956 ascii_encode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
957 PyObject *args)
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
958 {
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
959 PyObject *str, *v;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
960 const char *errors = NULL;
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
961
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
962 if (!PyArg_ParseTuple(args, "O|z:ascii_encode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
963 &str, &errors))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
964 return NULL;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
965
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
966 str = PyUnicode_FromObject(str);
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
967 if (str == NULL)
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
968 return NULL;
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
969 v = codec_tuple(PyUnicode_EncodeASCII(
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
970 PyUnicode_AS_UNICODE(str),
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
971 PyUnicode_GET_SIZE(str),
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
972 errors),
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
973 PyUnicode_GET_SIZE(str));
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
974 Py_DECREF(str);
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
975 return v;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
976 }
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
977
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
978 static PyObject *
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
979 charmap_encode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
980 PyObject *args)
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
981 {
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
982 PyObject *str, *v;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
983 const char *errors = NULL;
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
984 PyObject *mapping = NULL;
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
985
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
986 if (!PyArg_ParseTuple(args, "O|zO:charmap_encode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
987 &str, &errors, &mapping))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
988 return NULL;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
989 if (mapping == Py_None)
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
990 mapping = NULL;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
991
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
992 str = PyUnicode_FromObject(str);
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
993 if (str == NULL)
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
994 return NULL;
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
995 v = codec_tuple(PyUnicode_EncodeCharmap(
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
996 PyUnicode_AS_UNICODE(str),
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
997 PyUnicode_GET_SIZE(str),
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
998 mapping,
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
999 errors),
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1000 PyUnicode_GET_SIZE(str));
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
1001 Py_DECREF(str);
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
1002 return v;
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
1003 }
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
1004
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38669
diff changeset
1005 static PyObject*
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38669
diff changeset
1006 charmap_build(PyObject *self, PyObject *args)
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38669
diff changeset
1007 {
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38669
diff changeset
1008 PyObject *map;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38669
diff changeset
1009 if (!PyArg_ParseTuple(args, "U:charmap_build", &map))
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38669
diff changeset
1010 return NULL;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38669
diff changeset
1011 return PyUnicode_BuildEncodingMap(map);
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38669
diff changeset
1012 }
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38669
diff changeset
1013
71193
13e6d3cb2ecd Issue #9642: Uniformize the tests on the availability of the mbcs codec
Victor Stinner <victor.stinner@haypocalc.com>
parents: 68257
diff changeset
1014 #ifdef HAVE_MBCS
12074
65f35b73e350 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 11980
diff changeset
1015
65f35b73e350 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 11980
diff changeset
1016 static PyObject *
65f35b73e350 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 11980
diff changeset
1017 mbcs_encode(PyObject *self,
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1018 PyObject *args)
12074
65f35b73e350 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 11980
diff changeset
1019 {
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
1020 PyObject *str, *v;
12074
65f35b73e350 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 11980
diff changeset
1021 const char *errors = NULL;
65f35b73e350 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 11980
diff changeset
1022
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
1023 if (!PyArg_ParseTuple(args, "O|z:mbcs_encode",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1024 &str, &errors))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1025 return NULL;
12074
65f35b73e350 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 11980
diff changeset
1026
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
1027 str = PyUnicode_FromObject(str);
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
1028 if (str == NULL)
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1029 return NULL;
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
1030 v = codec_tuple(PyUnicode_EncodeMBCS(
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1031 PyUnicode_AS_UNICODE(str),
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1032 PyUnicode_GET_SIZE(str),
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1033 errors),
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1034 PyUnicode_GET_SIZE(str));
13358
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
1035 Py_DECREF(str);
c8706e5a882f Modified the encode interfaces of all builtin codecs to accept
Marc-André Lemburg <mal@egenix.com>
parents: 12074
diff changeset
1036 return v;
12074
65f35b73e350 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 11980
diff changeset
1037 }
65f35b73e350 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 11980
diff changeset
1038
72976
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
1039 static PyObject *
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
1040 code_page_encode(PyObject *self,
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
1041 PyObject *args)
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
1042 {
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
1043 PyObject *str, *v;
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
1044 const char *errors = NULL;
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
1045 int code_page;
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
1046
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
1047 if (!PyArg_ParseTuple(args, "iO|z:code_page_encode",
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
1048 &code_page, &str, &errors))
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
1049 return NULL;
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
1050
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
1051 str = PyUnicode_FromObject(str);
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
1052 if (str == NULL)
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
1053 return NULL;
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
1054 v = codec_tuple(PyUnicode_EncodeCodePage(code_page,
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
1055 str,
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
1056 errors),
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
1057 PyUnicode_GET_LENGTH(str));
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
1058 Py_DECREF(str);
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
1059 return v;
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
1060 }
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
1061
71193
13e6d3cb2ecd Issue #9642: Uniformize the tests on the availability of the mbcs codec
Victor Stinner <victor.stinner@haypocalc.com>
parents: 68257
diff changeset
1062 #endif /* HAVE_MBCS */
12074
65f35b73e350 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 11980
diff changeset
1063
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24808
diff changeset
1064 /* --- Error handler registry --------------------------------------------- */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24808
diff changeset
1065
25974
040af9c736bb Add docstrings to register, lookup, register_error
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
1066 PyDoc_STRVAR(register_error__doc__,
040af9c736bb Add docstrings to register, lookup, register_error
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
1067 "register_error(errors, handler)\n\
040af9c736bb Add docstrings to register, lookup, register_error
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
1068 \n\
040af9c736bb Add docstrings to register, lookup, register_error
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
1069 Register the specified error handler under the name\n\
040af9c736bb Add docstrings to register, lookup, register_error
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
1070 errors. handler must be a callable object, that\n\
040af9c736bb Add docstrings to register, lookup, register_error
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
1071 will be called with an exception instance containing\n\
040af9c736bb Add docstrings to register, lookup, register_error
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
1072 information about the location of the encoding/decoding\n\
040af9c736bb Add docstrings to register, lookup, register_error
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
1073 error and must return a (replacement, new position) tuple.");
040af9c736bb Add docstrings to register, lookup, register_error
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
1074
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24808
diff changeset
1075 static PyObject *register_error(PyObject *self, PyObject *args)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24808
diff changeset
1076 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24808
diff changeset
1077 const char *name;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24808
diff changeset
1078 PyObject *handler;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24808
diff changeset
1079
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24808
diff changeset
1080 if (!PyArg_ParseTuple(args, "sO:register_error",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1081 &name, &handler))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1082 return NULL;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24808
diff changeset
1083 if (PyCodec_RegisterError(name, handler))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24808
diff changeset
1084 return NULL;
38669
19fd6141940d Partially merge trunk into p3yk. The removal of Mac/Tools is confusing svn
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
1085 Py_RETURN_NONE;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24808
diff changeset
1086 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24808
diff changeset
1087
25974
040af9c736bb Add docstrings to register, lookup, register_error
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
1088 PyDoc_STRVAR(lookup_error__doc__,
040af9c736bb Add docstrings to register, lookup, register_error
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
1089 "lookup_error(errors) -> handler\n\
040af9c736bb Add docstrings to register, lookup, register_error
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
1090 \n\
040af9c736bb Add docstrings to register, lookup, register_error
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
1091 Return the error handler for the specified error handling name\n\
040af9c736bb Add docstrings to register, lookup, register_error
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
1092 or raise a LookupError, if no handler exists under this name.");
040af9c736bb Add docstrings to register, lookup, register_error
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
1093
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24808
diff changeset
1094 static PyObject *lookup_error(PyObject *self, PyObject *args)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24808
diff changeset
1095 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24808
diff changeset
1096 const char *name;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24808
diff changeset
1097
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24808
diff changeset
1098 if (!PyArg_ParseTuple(args, "s:lookup_error",
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1099 &name))
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1100 return NULL;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24808
diff changeset
1101 return PyCodec_LookupError(name);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24808
diff changeset
1102 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24808
diff changeset
1103
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
1104 /* --- Module API --------------------------------------------------------- */
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
1105
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
1106 static PyMethodDef _codecs_functions[] = {
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1107 {"register", codec_register, METH_O,
25974
040af9c736bb Add docstrings to register, lookup, register_error
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
1108 register__doc__},
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1109 {"lookup", codec_lookup, METH_VARARGS,
25974
040af9c736bb Add docstrings to register, lookup, register_error
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
1110 lookup__doc__},
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1111 {"encode", codec_encode, METH_VARARGS,
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1112 encode__doc__},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1113 {"decode", codec_decode, METH_VARARGS,
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1114 decode__doc__},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1115 {"escape_encode", escape_encode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1116 {"escape_decode", escape_decode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1117 {"utf_8_encode", utf_8_encode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1118 {"utf_8_decode", utf_8_decode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1119 {"utf_7_encode", utf_7_encode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1120 {"utf_7_decode", utf_7_decode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1121 {"utf_16_encode", utf_16_encode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1122 {"utf_16_le_encode", utf_16_le_encode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1123 {"utf_16_be_encode", utf_16_be_encode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1124 {"utf_16_decode", utf_16_decode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1125 {"utf_16_le_decode", utf_16_le_decode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1126 {"utf_16_be_decode", utf_16_be_decode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1127 {"utf_16_ex_decode", utf_16_ex_decode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1128 {"utf_32_encode", utf_32_encode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1129 {"utf_32_le_encode", utf_32_le_encode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1130 {"utf_32_be_encode", utf_32_be_encode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1131 {"utf_32_decode", utf_32_decode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1132 {"utf_32_le_decode", utf_32_le_decode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1133 {"utf_32_be_decode", utf_32_be_decode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1134 {"utf_32_ex_decode", utf_32_ex_decode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1135 {"unicode_escape_encode", unicode_escape_encode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1136 {"unicode_escape_decode", unicode_escape_decode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1137 {"unicode_internal_encode", unicode_internal_encode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1138 {"unicode_internal_decode", unicode_internal_decode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1139 {"raw_unicode_escape_encode", raw_unicode_escape_encode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1140 {"raw_unicode_escape_decode", raw_unicode_escape_decode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1141 {"latin_1_encode", latin_1_encode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1142 {"latin_1_decode", latin_1_decode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1143 {"ascii_encode", ascii_encode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1144 {"ascii_decode", ascii_decode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1145 {"charmap_encode", charmap_encode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1146 {"charmap_decode", charmap_decode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1147 {"charmap_build", charmap_build, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1148 {"readbuffer_encode", readbuffer_encode, METH_VARARGS},
71193
13e6d3cb2ecd Issue #9642: Uniformize the tests on the availability of the mbcs codec
Victor Stinner <victor.stinner@haypocalc.com>
parents: 68257
diff changeset
1149 #ifdef HAVE_MBCS
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1150 {"mbcs_encode", mbcs_encode, METH_VARARGS},
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1151 {"mbcs_decode", mbcs_decode, METH_VARARGS},
72976
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
1152 {"code_page_encode", code_page_encode, METH_VARARGS},
af0800b986b7 Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents: 72918
diff changeset
1153 {"code_page_decode", code_page_decode, METH_VARARGS},
12074
65f35b73e350 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 11980
diff changeset
1154 #endif
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1155 {"register_error", register_error, METH_VARARGS,
25974
040af9c736bb Add docstrings to register, lookup, register_error
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
1156 register_error__doc__},
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1157 {"lookup_error", lookup_error, METH_VARARGS,
25974
040af9c736bb Add docstrings to register, lookup, register_error
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
1158 lookup_error__doc__},
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1159 {NULL, NULL} /* sentinel */
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
1160 };
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
1161
47580
94f64f11fdd0 Implement PEP 3121: new module initialization and finalization API.
Martin v. Löwis <martin@v.loewis.de>
parents: 47301
diff changeset
1162 static struct PyModuleDef codecsmodule = {
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1163 PyModuleDef_HEAD_INIT,
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1164 "_codecs",
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1165 NULL,
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1166 -1,
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1167 _codecs_functions,
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1168 NULL,
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1169 NULL,
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1170 NULL,
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1171 NULL
47580
94f64f11fdd0 Implement PEP 3121: new module initialization and finalization API.
Martin v. Löwis <martin@v.loewis.de>
parents: 47301
diff changeset
1172 };
94f64f11fdd0 Implement PEP 3121: new module initialization and finalization API.
Martin v. Löwis <martin@v.loewis.de>
parents: 47301
diff changeset
1173
24471
f66b3a3509c7 Excise DL_EXPORT/DL_IMPORT from Modules/*. Required adding a prototype
Mark Hammond <mhammond@skippinet.com.au>
parents: 24071
diff changeset
1174 PyMODINIT_FUNC
47580
94f64f11fdd0 Implement PEP 3121: new module initialization and finalization API.
Martin v. Löwis <martin@v.loewis.de>
parents: 47301
diff changeset
1175 PyInit__codecs(void)
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
1176 {
61467
f197407bdba2 Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 53859
diff changeset
1177 return PyModule_Create(&codecsmodule);
11980
a9b5a31ff8cb Internal module _codecs -- Provides access to the codec registry and
Guido van Rossum <guido@python.org>
parents:
diff changeset
1178 }