Mercurial > cpython-lru_cache-weakref

annotate Python/codecs.c @ 85280:5e679ef2a55c 2.7

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Issue #13461: Fix a crash in the TextIOWrapper.tell method and in the "replace" error handler on 64-bit platforms. Patch by Yogesh Chaudhari.
author Serhiy Storchaka <storchaka@gmail.com>
date Tue, 20 Aug 2013 20:08:53 +0300
parents 719ee60fc5e2
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
1 /* ------------------------------------------------------------------------
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
2
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
3 Python Codec Registry and support functions
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
4
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
5 Written by Marc-Andre Lemburg (mal@lemburg.com).
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
6
13828
254f8b7c4a83 Changing the CNRI copyright notice according to CNRI's instructions.
Guido van Rossum <guido@python.org>
parents: 13658
diff changeset
7 Copyright (c) Corporation for National Research Initiatives.
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
8
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
9 ------------------------------------------------------------------------ */
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
10
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
11 #include "Python.h"
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
12 #include <ctype.h>
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
13
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
14 /* --- Codec Registry ----------------------------------------------------- */
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
15
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
16 /* Import the standard encodings package which will register the first
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
17 codec search function.
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
18
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
19 This is done in a lazy way so that the Unicode implementation does
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
20 not downgrade startup time of scripts not needing it.
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
21
12195
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup
Guido van Rossum <guido@python.org>
parents: 12048
diff changeset
22 ImportErrors are silently ignored by this function. Only one try is
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup
Guido van Rossum <guido@python.org>
parents: 12048
diff changeset
23 made.
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
24
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
25 */
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
26
28325
850626d47574 Fixed SF bug #663074. The codec system was using global static
Gustavo Niemeyer <gustavo@niemeyer.net>
parents: 27898
diff changeset
27 static int _PyCodecRegistry_Init(void); /* Forward */
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
28
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
29 int PyCodec_Register(PyObject *search_function)
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
30 {
31797
846b3ee7e106 Changed random calls to PyThreadState_Get() to use the macro
Nicholas Bastin <nick.bastin@gmail.com>
parents: 31273
diff changeset
31 PyInterpreterState *interp = PyThreadState_GET()->interp;
28325
850626d47574 Fixed SF bug #663074. The codec system was using global static
Gustavo Niemeyer <gustavo@niemeyer.net>
parents: 27898
diff changeset
32 if (interp->codec_search_path == NULL && _PyCodecRegistry_Init())
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
33 goto onError;
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
34 if (search_function == NULL) {
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
35 PyErr_BadArgument();
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
36 goto onError;
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
37 }
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
38 if (!PyCallable_Check(search_function)) {
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
39 PyErr_SetString(PyExc_TypeError, "argument must be callable");
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
40 goto onError;
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
41 }
28325
850626d47574 Fixed SF bug #663074. The codec system was using global static
Gustavo Niemeyer <gustavo@niemeyer.net>
parents: 27898
diff changeset
42 return PyList_Append(interp->codec_search_path, search_function);
12195
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup
Guido van Rossum <guido@python.org>
parents: 12048
diff changeset
43
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup
Guido van Rossum <guido@python.org>
parents: 12048
diff changeset
44 onError:
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup
Guido van Rossum <guido@python.org>
parents: 12048
diff changeset
45 return -1;
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
46 }
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
47
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12195
diff changeset
48 /* Convert a string to a normalized Python string: all characters are
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12195
diff changeset
49 converted to lower case, spaces are replaced with underscores. */
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12195
diff changeset
50
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
51 static
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12195
diff changeset
52 PyObject *normalizestring(const char *string)
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
53 {
13128
cb928d926f27 Change the loop index in normalizestring() to size_t too, to avoid a
Guido van Rossum <guido@python.org>
parents: 13096
diff changeset
54 register size_t i;
13096
8c8de5b736dc Trent Mick's Win64 changes: size_t vs. int or long; also some overflow
Guido van Rossum <guido@python.org>
parents: 12648
diff changeset
55 size_t len = strlen(string);
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
56 char *p;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
57 PyObject *v;
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
58
37673
606818c33e50 Replace INT_MAX with PY_SSIZE_T_MAX.
Martin v. Löwis <martin@v.loewis.de>
parents: 37076
diff changeset
59 if (len > PY_SSIZE_T_MAX) {
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
60 PyErr_SetString(PyExc_OverflowError, "string is too large");
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
61 return NULL;
37673
606818c33e50 Replace INT_MAX with PY_SSIZE_T_MAX.
Martin v. Löwis <martin@v.loewis.de>
parents: 37076
diff changeset
62 }
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
63
47527
52294af4b663 This reverts r63675 based on the discussion in this thread:
Gregory P. Smith <greg@mad-scientist.com>
parents: 47294
diff changeset
64 v = PyString_FromStringAndSize(NULL, len);
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
65 if (v == NULL)
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
66 return NULL;
47527
52294af4b663 This reverts r63675 based on the discussion in this thread:
Gregory P. Smith <greg@mad-scientist.com>
parents: 47294
diff changeset
67 p = PyString_AS_STRING(v);
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12195
diff changeset
68 for (i = 0; i < len; i++) {
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12195
diff changeset
69 register char ch = string[i];
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12195
diff changeset
70 if (ch == ' ')
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12195
diff changeset
71 ch = '-';
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12195
diff changeset
72 else
71492
739958134fe5 Issue #1813: Fix codec lookup and setting/getting locales under Turkish locales.
Antoine Pitrou <solipsis@pitrou.net>
parents: 61463
diff changeset
73 ch = Py_TOLOWER(Py_CHARMASK(ch));
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
74 p[i] = ch;
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12195
diff changeset
75 }
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
76 return v;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
77 }
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
78
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
79 /* Lookup the given encoding and return a tuple providing the codec
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
80 facilities.
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
81
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
82 The encoding string is looked up converted to all lower-case
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
83 characters. This makes encodings looked up through this mechanism
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
84 effectively case-insensitive.
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
85
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
86 If no codec is found, a LookupError is set and NULL returned.
12195
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup
Guido van Rossum <guido@python.org>
parents: 12048
diff changeset
87
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup
Guido van Rossum <guido@python.org>
parents: 12048
diff changeset
88 As side effect, this tries to load the encodings package, if not
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup
Guido van Rossum <guido@python.org>
parents: 12048
diff changeset
89 yet done. This is part of the lazy load strategy for the encodings
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup
Guido van Rossum <guido@python.org>
parents: 12048
diff changeset
90 package.
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup
Guido van Rossum <guido@python.org>
parents: 12048
diff changeset
91
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup
Guido van Rossum <guido@python.org>
parents: 12048
diff changeset
92 */
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
93
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
94 PyObject *_PyCodec_Lookup(const char *encoding)
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
95 {
28325
850626d47574 Fixed SF bug #663074. The codec system was using global static
Gustavo Niemeyer <gustavo@niemeyer.net>
parents: 27898
diff changeset
96 PyInterpreterState *interp;
12048
f87b56eebacc Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
97 PyObject *result, *args = NULL, *v;
37920
ce5585afc20f Fix more ssize_t issues.
Martin v. Löwis <martin@v.loewis.de>
parents: 37673
diff changeset
98 Py_ssize_t i, len;
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
99
12648
2a53f5dc0590 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12248
diff changeset
100 if (encoding == NULL) {
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
101 PyErr_BadArgument();
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
102 goto onError;
12648
2a53f5dc0590 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12248
diff changeset
103 }
28325
850626d47574 Fixed SF bug #663074. The codec system was using global static
Gustavo Niemeyer <gustavo@niemeyer.net>
parents: 27898
diff changeset
104
31797
846b3ee7e106 Changed random calls to PyThreadState_Get() to use the macro
Nicholas Bastin <nick.bastin@gmail.com>
parents: 31273
diff changeset
105 interp = PyThreadState_GET()->interp;
28325
850626d47574 Fixed SF bug #663074. The codec system was using global static
Gustavo Niemeyer <gustavo@niemeyer.net>
parents: 27898
diff changeset
106 if (interp->codec_search_path == NULL && _PyCodecRegistry_Init())
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
107 goto onError;
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
108
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12195
diff changeset
109 /* Convert the encoding to a normalized Python string: all
13620
5b727ab37092 Spelling fixes supplied by Rob W. W. Hooft. All these are fixes in either
Thomas Wouters <thomas@python.org>
parents: 13330
diff changeset
110 characters are converted to lower case, spaces and hyphens are
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12195
diff changeset
111 replaced with underscores. */
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12195
diff changeset
112 v = normalizestring(encoding);
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
113 if (v == NULL)
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
114 goto onError;
47527
52294af4b663 This reverts r63675 based on the discussion in this thread:
Gregory P. Smith <greg@mad-scientist.com>
parents: 47294
diff changeset
115 PyString_InternInPlace(&v);
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
116
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
117 /* First, try to lookup the name in the registry dictionary */
28325
850626d47574 Fixed SF bug #663074. The codec system was using global static
Gustavo Niemeyer <gustavo@niemeyer.net>
parents: 27898
diff changeset
118 result = PyDict_GetItem(interp->codec_search_cache, v);
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
119 if (result != NULL) {
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
120 Py_INCREF(result);
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
121 Py_DECREF(v);
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
122 return result;
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
123 }
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
124
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
125 /* Next, scan the search functions in order of registration */
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
126 args = PyTuple_New(1);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
127 if (args == NULL)
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
128 goto onError;
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
129 PyTuple_SET_ITEM(args,0,v);
12048
f87b56eebacc Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
130
28325
850626d47574 Fixed SF bug #663074. The codec system was using global static
Gustavo Niemeyer <gustavo@niemeyer.net>
parents: 27898
diff changeset
131 len = PyList_Size(interp->codec_search_path);
12048
f87b56eebacc Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
132 if (len < 0)
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
133 goto onError;
12195
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup
Guido van Rossum <guido@python.org>
parents: 12048
diff changeset
134 if (len == 0) {
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
135 PyErr_SetString(PyExc_LookupError,
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
136 "no codec search functions registered: "
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
137 "can't find encoding");
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
138 goto onError;
12195
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup
Guido van Rossum <guido@python.org>
parents: 12048
diff changeset
139 }
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
140
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
141 for (i = 0; i < len; i++) {
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
142 PyObject *func;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
143
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
144 func = PyList_GetItem(interp->codec_search_path, i);
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
145 if (func == NULL)
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
146 goto onError;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
147 result = PyEval_CallObject(func, args);
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
148 if (result == NULL)
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
149 goto onError;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
150 if (result == Py_None) {
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
151 Py_DECREF(result);
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
152 continue;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
153 }
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
154 if (!PyTuple_Check(result) || PyTuple_GET_SIZE(result) != 4) {
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
155 PyErr_SetString(PyExc_TypeError,
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
156 "codec search functions must return 4-tuples");
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
157 Py_DECREF(result);
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
158 goto onError;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
159 }
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
160 break;
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
161 }
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
162 if (i == len) {
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
163 /* XXX Perhaps we should cache misses too ? */
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
164 PyErr_Format(PyExc_LookupError,
25441
d32588c88633 Add encoding name in LookupError. Fixes #615013. Will backport to 2.2.
Martin v. Löwis <martin@v.loewis.de>
parents: 25095
diff changeset
165 "unknown encoding: %s", encoding);
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
166 goto onError;
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
167 }
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
168
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
169 /* Cache and return the result */
28325
850626d47574 Fixed SF bug #663074. The codec system was using global static
Gustavo Niemeyer <gustavo@niemeyer.net>
parents: 27898
diff changeset
170 PyDict_SetItem(interp->codec_search_cache, v, result);
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
171 Py_DECREF(args);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
172 return result;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
173
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
174 onError:
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
175 Py_XDECREF(args);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
176 return NULL;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
177 }
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
178
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
179 static
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
180 PyObject *args_tuple(PyObject *object,
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
181 const char *errors)
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
182 {
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
183 PyObject *args;
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
184
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
185 args = PyTuple_New(1 + (errors != NULL));
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
186 if (args == NULL)
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
187 return NULL;
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
188 Py_INCREF(object);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
189 PyTuple_SET_ITEM(args,0,object);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
190 if (errors) {
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
191 PyObject *v;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
192
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
193 v = PyString_FromString(errors);
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
194 if (v == NULL) {
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
195 Py_DECREF(args);
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
196 return NULL;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
197 }
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
198 PyTuple_SET_ITEM(args, 1, v);
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
199 }
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
200 return args;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
201 }
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
202
37036
c449944068d2 Fix wrong argument format in PyCodec_IncrementalEncoder() and
Walter Dörwald <walter@livinglogic.de>
parents: 36996
diff changeset
203 /* Helper function to get a codec item */
c449944068d2 Fix wrong argument format in PyCodec_IncrementalEncoder() and
Walter Dörwald <walter@livinglogic.de>
parents: 36996
diff changeset
204
c449944068d2 Fix wrong argument format in PyCodec_IncrementalEncoder() and
Walter Dörwald <walter@livinglogic.de>
parents: 36996
diff changeset
205 static
c449944068d2 Fix wrong argument format in PyCodec_IncrementalEncoder() and
Walter Dörwald <walter@livinglogic.de>
parents: 36996
diff changeset
206 PyObject *codec_getitem(const char *encoding, int index)
c449944068d2 Fix wrong argument format in PyCodec_IncrementalEncoder() and
Walter Dörwald <walter@livinglogic.de>
parents: 36996
diff changeset
207 {
c449944068d2 Fix wrong argument format in PyCodec_IncrementalEncoder() and
Walter Dörwald <walter@livinglogic.de>
parents: 36996
diff changeset
208 PyObject *codecs;
c449944068d2 Fix wrong argument format in PyCodec_IncrementalEncoder() and
Walter Dörwald <walter@livinglogic.de>
parents: 36996
diff changeset
209 PyObject *v;
c449944068d2 Fix wrong argument format in PyCodec_IncrementalEncoder() and
Walter Dörwald <walter@livinglogic.de>
parents: 36996
diff changeset
210
c449944068d2 Fix wrong argument format in PyCodec_IncrementalEncoder() and
Walter Dörwald <walter@livinglogic.de>
parents: 36996
diff changeset
211 codecs = _PyCodec_Lookup(encoding);
c449944068d2 Fix wrong argument format in PyCodec_IncrementalEncoder() and
Walter Dörwald <walter@livinglogic.de>
parents: 36996
diff changeset
212 if (codecs == NULL)
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
213 return NULL;
37036
c449944068d2 Fix wrong argument format in PyCodec_IncrementalEncoder() and
Walter Dörwald <walter@livinglogic.de>
parents: 36996
diff changeset
214 v = PyTuple_GET_ITEM(codecs, index);
c449944068d2 Fix wrong argument format in PyCodec_IncrementalEncoder() and
Walter Dörwald <walter@livinglogic.de>
parents: 36996
diff changeset
215 Py_DECREF(codecs);
c449944068d2 Fix wrong argument format in PyCodec_IncrementalEncoder() and
Walter Dörwald <walter@livinglogic.de>
parents: 36996
diff changeset
216 Py_INCREF(v);
c449944068d2 Fix wrong argument format in PyCodec_IncrementalEncoder() and
Walter Dörwald <walter@livinglogic.de>
parents: 36996
diff changeset
217 return v;
c449944068d2 Fix wrong argument format in PyCodec_IncrementalEncoder() and
Walter Dörwald <walter@livinglogic.de>
parents: 36996
diff changeset
218 }
c449944068d2 Fix wrong argument format in PyCodec_IncrementalEncoder() and
Walter Dörwald <walter@livinglogic.de>
parents: 36996
diff changeset
219
c449944068d2 Fix wrong argument format in PyCodec_IncrementalEncoder() and
Walter Dörwald <walter@livinglogic.de>
parents: 36996
diff changeset
220 /* Helper function to create an incremental codec. */
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
221
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
222 static
37036
c449944068d2 Fix wrong argument format in PyCodec_IncrementalEncoder() and
Walter Dörwald <walter@livinglogic.de>
parents: 36996
diff changeset
223 PyObject *codec_getincrementalcodec(const char *encoding,
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
224 const char *errors,
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
225 const char *attrname)
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
226 {
37036
c449944068d2 Fix wrong argument format in PyCodec_IncrementalEncoder() and
Walter Dörwald <walter@livinglogic.de>
parents: 36996
diff changeset
227 PyObject *codecs, *ret, *inccodec;
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
228
37036
c449944068d2 Fix wrong argument format in PyCodec_IncrementalEncoder() and
Walter Dörwald <walter@livinglogic.de>
parents: 36996
diff changeset
229 codecs = _PyCodec_Lookup(encoding);
c449944068d2 Fix wrong argument format in PyCodec_IncrementalEncoder() and
Walter Dörwald <walter@livinglogic.de>
parents: 36996
diff changeset
230 if (codecs == NULL)
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
231 return NULL;
37036
c449944068d2 Fix wrong argument format in PyCodec_IncrementalEncoder() and
Walter Dörwald <walter@livinglogic.de>
parents: 36996
diff changeset
232 inccodec = PyObject_GetAttrString(codecs, attrname);
37076
685e2123864d Release codecs variable earlier.
Walter Dörwald <walter@livinglogic.de>
parents: 37036
diff changeset
233 Py_DECREF(codecs);
685e2123864d Release codecs variable earlier.
Walter Dörwald <walter@livinglogic.de>
parents: 37036
diff changeset
234 if (inccodec == NULL)
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
235 return NULL;
37036
c449944068d2 Fix wrong argument format in PyCodec_IncrementalEncoder() and
Walter Dörwald <walter@livinglogic.de>
parents: 36996
diff changeset
236 if (errors)
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
237 ret = PyObject_CallFunction(inccodec, "s", errors);
37036
c449944068d2 Fix wrong argument format in PyCodec_IncrementalEncoder() and
Walter Dörwald <walter@livinglogic.de>
parents: 36996
diff changeset
238 else
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
239 ret = PyObject_CallFunction(inccodec, NULL);
37036
c449944068d2 Fix wrong argument format in PyCodec_IncrementalEncoder() and
Walter Dörwald <walter@livinglogic.de>
parents: 36996
diff changeset
240 Py_DECREF(inccodec);
c449944068d2 Fix wrong argument format in PyCodec_IncrementalEncoder() and
Walter Dörwald <walter@livinglogic.de>
parents: 36996
diff changeset
241 return ret;
c449944068d2 Fix wrong argument format in PyCodec_IncrementalEncoder() and
Walter Dörwald <walter@livinglogic.de>
parents: 36996
diff changeset
242 }
c449944068d2 Fix wrong argument format in PyCodec_IncrementalEncoder() and
Walter Dörwald <walter@livinglogic.de>
parents: 36996
diff changeset
243
c449944068d2 Fix wrong argument format in PyCodec_IncrementalEncoder() and
Walter Dörwald <walter@livinglogic.de>
parents: 36996
diff changeset
244 /* Helper function to create a stream codec. */
c449944068d2 Fix wrong argument format in PyCodec_IncrementalEncoder() and
Walter Dörwald <walter@livinglogic.de>
parents: 36996
diff changeset
245
c449944068d2 Fix wrong argument format in PyCodec_IncrementalEncoder() and
Walter Dörwald <walter@livinglogic.de>
parents: 36996
diff changeset
246 static
c449944068d2 Fix wrong argument format in PyCodec_IncrementalEncoder() and
Walter Dörwald <walter@livinglogic.de>
parents: 36996
diff changeset
247 PyObject *codec_getstreamcodec(const char *encoding,
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
248 PyObject *stream,
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
249 const char *errors,
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
250 const int index)
37036
c449944068d2 Fix wrong argument format in PyCodec_IncrementalEncoder() and
Walter Dörwald <walter@livinglogic.de>
parents: 36996
diff changeset
251 {
38923
6f042f5bf94d Bug #1511381: codec_getstreamcodec() in codec.c is corrected to
Hye-Shik Chang <hyeshik@gmail.com>
parents: 38023
diff changeset
252 PyObject *codecs, *streamcodec, *codeccls;
37036
c449944068d2 Fix wrong argument format in PyCodec_IncrementalEncoder() and
Walter Dörwald <walter@livinglogic.de>
parents: 36996
diff changeset
253
c449944068d2 Fix wrong argument format in PyCodec_IncrementalEncoder() and
Walter Dörwald <walter@livinglogic.de>
parents: 36996
diff changeset
254 codecs = _PyCodec_Lookup(encoding);
c449944068d2 Fix wrong argument format in PyCodec_IncrementalEncoder() and
Walter Dörwald <walter@livinglogic.de>
parents: 36996
diff changeset
255 if (codecs == NULL)
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
256 return NULL;
37036
c449944068d2 Fix wrong argument format in PyCodec_IncrementalEncoder() and
Walter Dörwald <walter@livinglogic.de>
parents: 36996
diff changeset
257
38923
6f042f5bf94d Bug #1511381: codec_getstreamcodec() in codec.c is corrected to
Hye-Shik Chang <hyeshik@gmail.com>
parents: 38023
diff changeset
258 codeccls = PyTuple_GET_ITEM(codecs, index);
6f042f5bf94d Bug #1511381: codec_getstreamcodec() in codec.c is corrected to
Hye-Shik Chang <hyeshik@gmail.com>
parents: 38023
diff changeset
259 if (errors != NULL)
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
260 streamcodec = PyObject_CallFunction(codeccls, "Os", stream, errors);
38923
6f042f5bf94d Bug #1511381: codec_getstreamcodec() in codec.c is corrected to
Hye-Shik Chang <hyeshik@gmail.com>
parents: 38023
diff changeset
261 else
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
262 streamcodec = PyObject_CallFunction(codeccls, "O", stream);
37036
c449944068d2 Fix wrong argument format in PyCodec_IncrementalEncoder() and
Walter Dörwald <walter@livinglogic.de>
parents: 36996
diff changeset
263 Py_DECREF(codecs);
c449944068d2 Fix wrong argument format in PyCodec_IncrementalEncoder() and
Walter Dörwald <walter@livinglogic.de>
parents: 36996
diff changeset
264 return streamcodec;
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
265 }
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
266
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
267 /* Convenience APIs to query the Codec registry.
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
268
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
269 All APIs return a codec object with incremented refcount.
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
270
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
271 */
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
272
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
273 PyObject *PyCodec_Encoder(const char *encoding)
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
274 {
37036
c449944068d2 Fix wrong argument format in PyCodec_IncrementalEncoder() and
Walter Dörwald <walter@livinglogic.de>
parents: 36996
diff changeset
275 return codec_getitem(encoding, 0);
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
276 }
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
277
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
278 PyObject *PyCodec_Decoder(const char *encoding)
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
279 {
37036
c449944068d2 Fix wrong argument format in PyCodec_IncrementalEncoder() and
Walter Dörwald <walter@livinglogic.de>
parents: 36996
diff changeset
280 return codec_getitem(encoding, 1);
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
281 }
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
282
36996
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass
Walter Dörwald <walter@livinglogic.de>
parents: 36751
diff changeset
283 PyObject *PyCodec_IncrementalEncoder(const char *encoding,
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
284 const char *errors)
36996
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass
Walter Dörwald <walter@livinglogic.de>
parents: 36751
diff changeset
285 {
37036
c449944068d2 Fix wrong argument format in PyCodec_IncrementalEncoder() and
Walter Dörwald <walter@livinglogic.de>
parents: 36996
diff changeset
286 return codec_getincrementalcodec(encoding, errors, "incrementalencoder");
36996
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass
Walter Dörwald <walter@livinglogic.de>
parents: 36751
diff changeset
287 }
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass
Walter Dörwald <walter@livinglogic.de>
parents: 36751
diff changeset
288
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass
Walter Dörwald <walter@livinglogic.de>
parents: 36751
diff changeset
289 PyObject *PyCodec_IncrementalDecoder(const char *encoding,
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
290 const char *errors)
36996
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass
Walter Dörwald <walter@livinglogic.de>
parents: 36751
diff changeset
291 {
37036
c449944068d2 Fix wrong argument format in PyCodec_IncrementalEncoder() and
Walter Dörwald <walter@livinglogic.de>
parents: 36996
diff changeset
292 return codec_getincrementalcodec(encoding, errors, "incrementaldecoder");
36996
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass
Walter Dörwald <walter@livinglogic.de>
parents: 36751
diff changeset
293 }
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass
Walter Dörwald <walter@livinglogic.de>
parents: 36751
diff changeset
294
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
295 PyObject *PyCodec_StreamReader(const char *encoding,
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
296 PyObject *stream,
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
297 const char *errors)
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
298 {
37036
c449944068d2 Fix wrong argument format in PyCodec_IncrementalEncoder() and
Walter Dörwald <walter@livinglogic.de>
parents: 36996
diff changeset
299 return codec_getstreamcodec(encoding, stream, errors, 2);
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
300 }
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
301
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
302 PyObject *PyCodec_StreamWriter(const char *encoding,
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
303 PyObject *stream,
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
304 const char *errors)
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
305 {
37036
c449944068d2 Fix wrong argument format in PyCodec_IncrementalEncoder() and
Walter Dörwald <walter@livinglogic.de>
parents: 36996
diff changeset
306 return codec_getstreamcodec(encoding, stream, errors, 3);
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
307 }
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
308
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
309 /* Encode an object (e.g. an Unicode object) using the given encoding
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
310 and return the resulting encoded object (usually a Python string).
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
311
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
312 errors is passed to the encoder factory as argument if non-NULL. */
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
313
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
314 PyObject *PyCodec_Encode(PyObject *object,
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
315 const char *encoding,
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
316 const char *errors)
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
317 {
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
318 PyObject *encoder = NULL;
35939
79374eb6db51 Fix a few more ref leaks. Backport candidate
Neal Norwitz <nnorwitz@gmail.com>
parents: 32533
diff changeset
319 PyObject *args = NULL, *result = NULL;
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
320 PyObject *v;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
321
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
322 encoder = PyCodec_Encoder(encoding);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
323 if (encoder == NULL)
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
324 goto onError;
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
325
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
326 args = args_tuple(object, errors);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
327 if (args == NULL)
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
328 goto onError;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
329
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
330 result = PyEval_CallObject(encoder,args);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
331 if (result == NULL)
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
332 goto onError;
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
333
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
334 if (!PyTuple_Check(result) ||
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
335 PyTuple_GET_SIZE(result) != 2) {
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
336 PyErr_SetString(PyExc_TypeError,
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
337 "encoder must return a tuple (object,integer)");
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
338 goto onError;
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
339 }
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
340 v = PyTuple_GET_ITEM(result,0);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
341 Py_INCREF(v);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
342 /* We don't check or use the second (integer) entry. */
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
343
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
344 Py_DECREF(args);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
345 Py_DECREF(encoder);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
346 Py_DECREF(result);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
347 return v;
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
348
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
349 onError:
35939
79374eb6db51 Fix a few more ref leaks. Backport candidate
Neal Norwitz <nnorwitz@gmail.com>
parents: 32533
diff changeset
350 Py_XDECREF(result);
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
351 Py_XDECREF(args);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
352 Py_XDECREF(encoder);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
353 return NULL;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
354 }
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
355
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
356 /* Decode an object (usually a Python string) using the given encoding
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
357 and return an equivalent object (e.g. an Unicode object).
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
358
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
359 errors is passed to the decoder factory as argument if non-NULL. */
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
360
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
361 PyObject *PyCodec_Decode(PyObject *object,
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
362 const char *encoding,
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
363 const char *errors)
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
364 {
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
365 PyObject *decoder = NULL;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
366 PyObject *args = NULL, *result = NULL;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
367 PyObject *v;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
368
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
369 decoder = PyCodec_Decoder(encoding);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
370 if (decoder == NULL)
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
371 goto onError;
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
372
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
373 args = args_tuple(object, errors);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
374 if (args == NULL)
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
375 goto onError;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
376
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
377 result = PyEval_CallObject(decoder,args);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
378 if (result == NULL)
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
379 goto onError;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
380 if (!PyTuple_Check(result) ||
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
381 PyTuple_GET_SIZE(result) != 2) {
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
382 PyErr_SetString(PyExc_TypeError,
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
383 "decoder must return a tuple (object,integer)");
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
384 goto onError;
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
385 }
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
386 v = PyTuple_GET_ITEM(result,0);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
387 Py_INCREF(v);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
388 /* We don't check or use the second (integer) entry. */
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
389
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
390 Py_DECREF(args);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
391 Py_DECREF(decoder);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
392 Py_DECREF(result);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
393 return v;
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
394
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
395 onError:
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
396 Py_XDECREF(args);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
397 Py_XDECREF(decoder);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
398 Py_XDECREF(result);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
399 return NULL;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
400 }
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
401
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
402 /* Register the error handling callback function error under the name
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
403 name. This function will be called by the codec when it encounters
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
404 an unencodable characters/undecodable bytes and doesn't know the
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
405 callback name, when name is specified as the error parameter
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
406 in the call to the encode/decode function.
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
407 Return 0 on success, -1 on error */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
408 int PyCodec_RegisterError(const char *name, PyObject *error)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
409 {
31797
846b3ee7e106 Changed random calls to PyThreadState_Get() to use the macro
Nicholas Bastin <nick.bastin@gmail.com>
parents: 31273
diff changeset
410 PyInterpreterState *interp = PyThreadState_GET()->interp;
28325
850626d47574 Fixed SF bug #663074. The codec system was using global static
Gustavo Niemeyer <gustavo@niemeyer.net>
parents: 27898
diff changeset
411 if (interp->codec_search_path == NULL && _PyCodecRegistry_Init())
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
412 return -1;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
413 if (!PyCallable_Check(error)) {
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
414 PyErr_SetString(PyExc_TypeError, "handler must be callable");
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
415 return -1;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
416 }
28325
850626d47574 Fixed SF bug #663074. The codec system was using global static
Gustavo Niemeyer <gustavo@niemeyer.net>
parents: 27898
diff changeset
417 return PyDict_SetItemString(interp->codec_error_registry,
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
418 (char *)name, error);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
419 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
420
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
421 /* Lookup the error handling callback function registered under the
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
422 name error. As a special case NULL can be passed, in which case
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
423 the error handling callback for strict encoding will be returned. */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
424 PyObject *PyCodec_LookupError(const char *name)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
425 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
426 PyObject *handler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
427
31797
846b3ee7e106 Changed random calls to PyThreadState_Get() to use the macro
Nicholas Bastin <nick.bastin@gmail.com>
parents: 31273
diff changeset
428 PyInterpreterState *interp = PyThreadState_GET()->interp;
28325
850626d47574 Fixed SF bug #663074. The codec system was using global static
Gustavo Niemeyer <gustavo@niemeyer.net>
parents: 27898
diff changeset
429 if (interp->codec_search_path == NULL && _PyCodecRegistry_Init())
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
430 return NULL;
28325
850626d47574 Fixed SF bug #663074. The codec system was using global static
Gustavo Niemeyer <gustavo@niemeyer.net>
parents: 27898
diff changeset
431
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
432 if (name==NULL)
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
433 name = "strict";
28325
850626d47574 Fixed SF bug #663074. The codec system was using global static
Gustavo Niemeyer <gustavo@niemeyer.net>
parents: 27898
diff changeset
434 handler = PyDict_GetItemString(interp->codec_error_registry, (char *)name);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
435 if (!handler)
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
436 PyErr_Format(PyExc_LookupError, "unknown error handler name '%.400s'", name);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
437 else
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
438 Py_INCREF(handler);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
439 return handler;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
440 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
441
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
442 static void wrong_exception_type(PyObject *exc)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
443 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
444 PyObject *type = PyObject_GetAttrString(exc, "__class__");
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
445 if (type != NULL) {
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
446 PyObject *name = PyObject_GetAttrString(type, "__name__");
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
447 Py_DECREF(type);
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
448 if (name != NULL) {
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
449 PyObject *string = PyObject_Str(name);
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
450 Py_DECREF(name);
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
451 if (string != NULL) {
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
452 PyErr_Format(PyExc_TypeError,
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
453 "don't know how to handle %.400s in error callback",
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
454 PyString_AS_STRING(string));
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
455 Py_DECREF(string);
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
456 }
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
457 }
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
458 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
459 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
460
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
461 PyObject *PyCodec_StrictErrors(PyObject *exc)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
462 {
36751
8e04de2480e4 PEP 352 implementation. Creates a new base class, BaseException, which has an
Brett Cannon <bcannon@gmail.com>
parents: 36712
diff changeset
463 if (PyExceptionInstance_Check(exc))
8e04de2480e4 PEP 352 implementation. Creates a new base class, BaseException, which has an
Brett Cannon <bcannon@gmail.com>
parents: 36712
diff changeset
464 PyErr_SetObject(PyExceptionInstance_Class(exc), exc);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
465 else
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
466 PyErr_SetString(PyExc_TypeError, "codec must pass exception instance");
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
467 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
468 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
469
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
470
26318
910fa739f4a4 Fix PEP 293 related problems with --disable-unicode builds
Walter Dörwald <walter@livinglogic.de>
parents: 25441
diff changeset
471 #ifdef Py_USING_UNICODE
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
472 PyObject *PyCodec_IgnoreErrors(PyObject *exc)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
473 {
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 35939
diff changeset
474 Py_ssize_t end;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
475 if (PyObject_IsInstance(exc, PyExc_UnicodeEncodeError)) {
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
476 if (PyUnicodeEncodeError_GetEnd(exc, &end))
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
477 return NULL;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
478 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
479 else if (PyObject_IsInstance(exc, PyExc_UnicodeDecodeError)) {
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
480 if (PyUnicodeDecodeError_GetEnd(exc, &end))
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
481 return NULL;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
482 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
483 else if (PyObject_IsInstance(exc, PyExc_UnicodeTranslateError)) {
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
484 if (PyUnicodeTranslateError_GetEnd(exc, &end))
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
485 return NULL;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
486 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
487 else {
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
488 wrong_exception_type(exc);
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
489 return NULL;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
490 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
491 /* ouch: passing NULL, 0, pos gives None instead of u'' */
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 35939
diff changeset
492 return Py_BuildValue("(u#n)", &end, 0, end);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
493 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
494
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
495
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
496 PyObject *PyCodec_ReplaceErrors(PyObject *exc)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
497 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
498 PyObject *restuple;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 35939
diff changeset
499 Py_ssize_t start;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 35939
diff changeset
500 Py_ssize_t end;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 35939
diff changeset
501 Py_ssize_t i;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
502
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
503 if (PyObject_IsInstance(exc, PyExc_UnicodeEncodeError)) {
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
504 PyObject *res;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
505 Py_UNICODE *p;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
506 if (PyUnicodeEncodeError_GetStart(exc, &start))
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
507 return NULL;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
508 if (PyUnicodeEncodeError_GetEnd(exc, &end))
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
509 return NULL;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
510 res = PyUnicode_FromUnicode(NULL, end-start);
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
511 if (res == NULL)
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
512 return NULL;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
513 for (p = PyUnicode_AS_UNICODE(res), i = start;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
514 i<end; ++p, ++i)
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
515 *p = '?';
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
516 restuple = Py_BuildValue("(On)", res, end);
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
517 Py_DECREF(res);
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
518 return restuple;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
519 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
520 else if (PyObject_IsInstance(exc, PyExc_UnicodeDecodeError)) {
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
521 Py_UNICODE res = Py_UNICODE_REPLACEMENT_CHARACTER;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
522 if (PyUnicodeDecodeError_GetEnd(exc, &end))
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
523 return NULL;
85280
5e679ef2a55c Issue #13461: Fix a crash in the TextIOWrapper.tell method and in the "replace"
Serhiy Storchaka <storchaka@gmail.com>
parents: 85051
diff changeset
524 return Py_BuildValue("(u#n)", &res, (Py_ssize_t)1, end);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
525 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
526 else if (PyObject_IsInstance(exc, PyExc_UnicodeTranslateError)) {
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
527 PyObject *res;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
528 Py_UNICODE *p;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
529 if (PyUnicodeTranslateError_GetStart(exc, &start))
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
530 return NULL;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
531 if (PyUnicodeTranslateError_GetEnd(exc, &end))
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
532 return NULL;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
533 res = PyUnicode_FromUnicode(NULL, end-start);
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
534 if (res == NULL)
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
535 return NULL;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
536 for (p = PyUnicode_AS_UNICODE(res), i = start;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
537 i<end; ++p, ++i)
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
538 *p = Py_UNICODE_REPLACEMENT_CHARACTER;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
539 restuple = Py_BuildValue("(On)", res, end);
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
540 Py_DECREF(res);
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
541 return restuple;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
542 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
543 else {
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
544 wrong_exception_type(exc);
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
545 return NULL;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
546 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
547 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
548
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
549 PyObject *PyCodec_XMLCharRefReplaceErrors(PyObject *exc)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
550 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
551 if (PyObject_IsInstance(exc, PyExc_UnicodeEncodeError)) {
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
552 PyObject *restuple;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
553 PyObject *object;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
554 Py_ssize_t start;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
555 Py_ssize_t end;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
556 PyObject *res;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
557 Py_UNICODE *p;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
558 Py_UNICODE *startp;
85051
719ee60fc5e2 Issue #15866: The xmlcharrefreplace error handler no more produces two XML
Serhiy Storchaka <storchaka@gmail.com>
parents: 71492
diff changeset
559 Py_UNICODE *e;
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
560 Py_UNICODE *outp;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
561 int ressize;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
562 if (PyUnicodeEncodeError_GetStart(exc, &start))
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
563 return NULL;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
564 if (PyUnicodeEncodeError_GetEnd(exc, &end))
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
565 return NULL;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
566 if (!(object = PyUnicodeEncodeError_GetObject(exc)))
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
567 return NULL;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
568 startp = PyUnicode_AS_UNICODE(object);
85051
719ee60fc5e2 Issue #15866: The xmlcharrefreplace error handler no more produces two XML
Serhiy Storchaka <storchaka@gmail.com>
parents: 71492
diff changeset
569 e = startp + end;
719ee60fc5e2 Issue #15866: The xmlcharrefreplace error handler no more produces two XML
Serhiy Storchaka <storchaka@gmail.com>
parents: 71492
diff changeset
570 for (p = startp+start, ressize = 0; p < e;) {
719ee60fc5e2 Issue #15866: The xmlcharrefreplace error handler no more produces two XML
Serhiy Storchaka <storchaka@gmail.com>
parents: 71492
diff changeset
571 Py_UCS4 ch = *p++;
719ee60fc5e2 Issue #15866: The xmlcharrefreplace error handler no more produces two XML
Serhiy Storchaka <storchaka@gmail.com>
parents: 71492
diff changeset
572 #ifndef Py_UNICODE_WIDE
719ee60fc5e2 Issue #15866: The xmlcharrefreplace error handler no more produces two XML
Serhiy Storchaka <storchaka@gmail.com>
parents: 71492
diff changeset
573 if ((0xD800 <= ch && ch <= 0xDBFF) &&
719ee60fc5e2 Issue #15866: The xmlcharrefreplace error handler no more produces two XML
Serhiy Storchaka <storchaka@gmail.com>
parents: 71492
diff changeset
574 (p < e) &&
719ee60fc5e2 Issue #15866: The xmlcharrefreplace error handler no more produces two XML
Serhiy Storchaka <storchaka@gmail.com>
parents: 71492
diff changeset
575 (0xDC00 <= *p && *p <= 0xDFFF)) {
719ee60fc5e2 Issue #15866: The xmlcharrefreplace error handler no more produces two XML
Serhiy Storchaka <storchaka@gmail.com>
parents: 71492
diff changeset
576 ch = ((((ch & 0x03FF) << 10) |
719ee60fc5e2 Issue #15866: The xmlcharrefreplace error handler no more produces two XML
Serhiy Storchaka <storchaka@gmail.com>
parents: 71492
diff changeset
577 ((Py_UCS4)*p++ & 0x03FF)) + 0x10000);
719ee60fc5e2 Issue #15866: The xmlcharrefreplace error handler no more produces two XML
Serhiy Storchaka <storchaka@gmail.com>
parents: 71492
diff changeset
578 }
719ee60fc5e2 Issue #15866: The xmlcharrefreplace error handler no more produces two XML
Serhiy Storchaka <storchaka@gmail.com>
parents: 71492
diff changeset
579 #endif
719ee60fc5e2 Issue #15866: The xmlcharrefreplace error handler no more produces two XML
Serhiy Storchaka <storchaka@gmail.com>
parents: 71492
diff changeset
580 if (ch < 10)
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
581 ressize += 2+1+1;
85051
719ee60fc5e2 Issue #15866: The xmlcharrefreplace error handler no more produces two XML
Serhiy Storchaka <storchaka@gmail.com>
parents: 71492
diff changeset
582 else if (ch < 100)
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
583 ressize += 2+2+1;
85051
719ee60fc5e2 Issue #15866: The xmlcharrefreplace error handler no more produces two XML
Serhiy Storchaka <storchaka@gmail.com>
parents: 71492
diff changeset
584 else if (ch < 1000)
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
585 ressize += 2+3+1;
85051
719ee60fc5e2 Issue #15866: The xmlcharrefreplace error handler no more produces two XML
Serhiy Storchaka <storchaka@gmail.com>
parents: 71492
diff changeset
586 else if (ch < 10000)
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
587 ressize += 2+4+1;
85051
719ee60fc5e2 Issue #15866: The xmlcharrefreplace error handler no more produces two XML
Serhiy Storchaka <storchaka@gmail.com>
parents: 71492
diff changeset
588 else if (ch < 100000)
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
589 ressize += 2+5+1;
85051
719ee60fc5e2 Issue #15866: The xmlcharrefreplace error handler no more produces two XML
Serhiy Storchaka <storchaka@gmail.com>
parents: 71492
diff changeset
590 else if (ch < 1000000)
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
591 ressize += 2+6+1;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
592 else
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
593 ressize += 2+7+1;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
594 }
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
595 /* allocate replacement */
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
596 res = PyUnicode_FromUnicode(NULL, ressize);
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
597 if (res == NULL) {
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
598 Py_DECREF(object);
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
599 return NULL;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
600 }
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
601 /* generate replacement */
85051
719ee60fc5e2 Issue #15866: The xmlcharrefreplace error handler no more produces two XML
Serhiy Storchaka <storchaka@gmail.com>
parents: 71492
diff changeset
602 for (p = startp+start, outp = PyUnicode_AS_UNICODE(res); p < e;) {
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
603 int digits;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
604 int base;
85051
719ee60fc5e2 Issue #15866: The xmlcharrefreplace error handler no more produces two XML
Serhiy Storchaka <storchaka@gmail.com>
parents: 71492
diff changeset
605 Py_UCS4 ch = *p++;
719ee60fc5e2 Issue #15866: The xmlcharrefreplace error handler no more produces two XML
Serhiy Storchaka <storchaka@gmail.com>
parents: 71492
diff changeset
606 #ifndef Py_UNICODE_WIDE
719ee60fc5e2 Issue #15866: The xmlcharrefreplace error handler no more produces two XML
Serhiy Storchaka <storchaka@gmail.com>
parents: 71492
diff changeset
607 if ((0xD800 <= ch && ch <= 0xDBFF) &&
719ee60fc5e2 Issue #15866: The xmlcharrefreplace error handler no more produces two XML
Serhiy Storchaka <storchaka@gmail.com>
parents: 71492
diff changeset
608 (p < startp+end) &&
719ee60fc5e2 Issue #15866: The xmlcharrefreplace error handler no more produces two XML
Serhiy Storchaka <storchaka@gmail.com>
parents: 71492
diff changeset
609 (0xDC00 <= *p && *p <= 0xDFFF)) {
719ee60fc5e2 Issue #15866: The xmlcharrefreplace error handler no more produces two XML
Serhiy Storchaka <storchaka@gmail.com>
parents: 71492
diff changeset
610 ch = ((((ch & 0x03FF) << 10) |
719ee60fc5e2 Issue #15866: The xmlcharrefreplace error handler no more produces two XML
Serhiy Storchaka <storchaka@gmail.com>
parents: 71492
diff changeset
611 ((Py_UCS4)*p++ & 0x03FF)) + 0x10000);
719ee60fc5e2 Issue #15866: The xmlcharrefreplace error handler no more produces two XML
Serhiy Storchaka <storchaka@gmail.com>
parents: 71492
diff changeset
612 }
719ee60fc5e2 Issue #15866: The xmlcharrefreplace error handler no more produces two XML
Serhiy Storchaka <storchaka@gmail.com>
parents: 71492
diff changeset
613 #endif
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
614 *outp++ = '&';
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
615 *outp++ = '#';
85051
719ee60fc5e2 Issue #15866: The xmlcharrefreplace error handler no more produces two XML
Serhiy Storchaka <storchaka@gmail.com>
parents: 71492
diff changeset
616 if (ch < 10) {
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
617 digits = 1;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
618 base = 1;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
619 }
85051
719ee60fc5e2 Issue #15866: The xmlcharrefreplace error handler no more produces two XML
Serhiy Storchaka <storchaka@gmail.com>
parents: 71492
diff changeset
620 else if (ch < 100) {
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
621 digits = 2;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
622 base = 10;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
623 }
85051
719ee60fc5e2 Issue #15866: The xmlcharrefreplace error handler no more produces two XML
Serhiy Storchaka <storchaka@gmail.com>
parents: 71492
diff changeset
624 else if (ch < 1000) {
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
625 digits = 3;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
626 base = 100;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
627 }
85051
719ee60fc5e2 Issue #15866: The xmlcharrefreplace error handler no more produces two XML
Serhiy Storchaka <storchaka@gmail.com>
parents: 71492
diff changeset
628 else if (ch < 10000) {
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
629 digits = 4;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
630 base = 1000;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
631 }
85051
719ee60fc5e2 Issue #15866: The xmlcharrefreplace error handler no more produces two XML
Serhiy Storchaka <storchaka@gmail.com>
parents: 71492
diff changeset
632 else if (ch < 100000) {
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
633 digits = 5;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
634 base = 10000;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
635 }
85051
719ee60fc5e2 Issue #15866: The xmlcharrefreplace error handler no more produces two XML
Serhiy Storchaka <storchaka@gmail.com>
parents: 71492
diff changeset
636 else if (ch < 1000000) {
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
637 digits = 6;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
638 base = 100000;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
639 }
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
640 else {
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
641 digits = 7;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
642 base = 1000000;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
643 }
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
644 while (digits-->0) {
85051
719ee60fc5e2 Issue #15866: The xmlcharrefreplace error handler no more produces two XML
Serhiy Storchaka <storchaka@gmail.com>
parents: 71492
diff changeset
645 *outp++ = '0' + ch/base;
719ee60fc5e2 Issue #15866: The xmlcharrefreplace error handler no more produces two XML
Serhiy Storchaka <storchaka@gmail.com>
parents: 71492
diff changeset
646 ch %= base;
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
647 base /= 10;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
648 }
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
649 *outp++ = ';';
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
650 }
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
651 restuple = Py_BuildValue("(On)", res, end);
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
652 Py_DECREF(res);
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
653 Py_DECREF(object);
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
654 return restuple;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
655 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
656 else {
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
657 wrong_exception_type(exc);
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
658 return NULL;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
659 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
660 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
661
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
662 static Py_UNICODE hexdigits[] = {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
663 '0', '1', '2', '3', '4', '5', '6', '7',
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
664 '8', '9', 'a', 'b', 'c', 'd', 'e', 'f'
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
665 };
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
666
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
667 PyObject *PyCodec_BackslashReplaceErrors(PyObject *exc)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
668 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
669 if (PyObject_IsInstance(exc, PyExc_UnicodeEncodeError)) {
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
670 PyObject *restuple;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
671 PyObject *object;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
672 Py_ssize_t start;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
673 Py_ssize_t end;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
674 PyObject *res;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
675 Py_UNICODE *p;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
676 Py_UNICODE *startp;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
677 Py_UNICODE *outp;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
678 int ressize;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
679 if (PyUnicodeEncodeError_GetStart(exc, &start))
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
680 return NULL;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
681 if (PyUnicodeEncodeError_GetEnd(exc, &end))
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
682 return NULL;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
683 if (!(object = PyUnicodeEncodeError_GetObject(exc)))
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
684 return NULL;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
685 startp = PyUnicode_AS_UNICODE(object);
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
686 for (p = startp+start, ressize = 0; p < startp+end; ++p) {
31273
25519d6c97b8 Fix gcc 3.3 warnings related to Py_UNICODE_WIDE.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 28325
diff changeset
687 #ifdef Py_UNICODE_WIDE
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
688 if (*p >= 0x00010000)
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
689 ressize += 1+1+8;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
690 else
31273
25519d6c97b8 Fix gcc 3.3 warnings related to Py_UNICODE_WIDE.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 28325
diff changeset
691 #endif
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
692 if (*p >= 0x100) {
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
693 ressize += 1+1+4;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
694 }
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
695 else
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
696 ressize += 1+1+2;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
697 }
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
698 res = PyUnicode_FromUnicode(NULL, ressize);
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
699 if (res==NULL)
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
700 return NULL;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
701 for (p = startp+start, outp = PyUnicode_AS_UNICODE(res);
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
702 p < startp+end; ++p) {
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
703 Py_UNICODE c = *p;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
704 *outp++ = '\\';
31273
25519d6c97b8 Fix gcc 3.3 warnings related to Py_UNICODE_WIDE.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 28325
diff changeset
705 #ifdef Py_UNICODE_WIDE
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
706 if (c >= 0x00010000) {
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
707 *outp++ = 'U';
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
708 *outp++ = hexdigits[(c>>28)&0xf];
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
709 *outp++ = hexdigits[(c>>24)&0xf];
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
710 *outp++ = hexdigits[(c>>20)&0xf];
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
711 *outp++ = hexdigits[(c>>16)&0xf];
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
712 *outp++ = hexdigits[(c>>12)&0xf];
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
713 *outp++ = hexdigits[(c>>8)&0xf];
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
714 }
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
715 else
31273
25519d6c97b8 Fix gcc 3.3 warnings related to Py_UNICODE_WIDE.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 28325
diff changeset
716 #endif
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
717 if (c >= 0x100) {
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
718 *outp++ = 'u';
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
719 *outp++ = hexdigits[(c>>12)&0xf];
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
720 *outp++ = hexdigits[(c>>8)&0xf];
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
721 }
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
722 else
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
723 *outp++ = 'x';
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
724 *outp++ = hexdigits[(c>>4)&0xf];
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
725 *outp++ = hexdigits[c&0xf];
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
726 }
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
727
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
728 restuple = Py_BuildValue("(On)", res, end);
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
729 Py_DECREF(res);
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
730 Py_DECREF(object);
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
731 return restuple;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
732 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
733 else {
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
734 wrong_exception_type(exc);
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
735 return NULL;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
736 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
737 }
26318
910fa739f4a4 Fix PEP 293 related problems with --disable-unicode builds
Walter Dörwald <walter@livinglogic.de>
parents: 25441
diff changeset
738 #endif
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
739
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
740 static PyObject *strict_errors(PyObject *self, PyObject *exc)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
741 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
742 return PyCodec_StrictErrors(exc);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
743 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
744
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
745
26318
910fa739f4a4 Fix PEP 293 related problems with --disable-unicode builds
Walter Dörwald <walter@livinglogic.de>
parents: 25441
diff changeset
746 #ifdef Py_USING_UNICODE
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
747 static PyObject *ignore_errors(PyObject *self, PyObject *exc)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
748 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
749 return PyCodec_IgnoreErrors(exc);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
750 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
751
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
752
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
753 static PyObject *replace_errors(PyObject *self, PyObject *exc)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
754 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
755 return PyCodec_ReplaceErrors(exc);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
756 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
757
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
758
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
759 static PyObject *xmlcharrefreplace_errors(PyObject *self, PyObject *exc)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
760 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
761 return PyCodec_XMLCharRefReplaceErrors(exc);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
762 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
763
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
764
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
765 static PyObject *backslashreplace_errors(PyObject *self, PyObject *exc)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
766 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
767 return PyCodec_BackslashReplaceErrors(exc);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
768 }
26318
910fa739f4a4 Fix PEP 293 related problems with --disable-unicode builds
Walter Dörwald <walter@livinglogic.de>
parents: 25441
diff changeset
769 #endif
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
770
28325
850626d47574 Fixed SF bug #663074. The codec system was using global static
Gustavo Niemeyer <gustavo@niemeyer.net>
parents: 27898
diff changeset
771 static int _PyCodecRegistry_Init(void)
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
772 {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
773 static struct {
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
774 char *name;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
775 PyMethodDef def;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
776 } methods[] =
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
777 {
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
778 {
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
779 "strict",
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
780 {
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
781 "strict_errors",
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
782 strict_errors,
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
783 METH_O,
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
784 PyDoc_STR("Implements the 'strict' error handling, which "
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
785 "raises a UnicodeError on coding errors.")
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
786 }
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
787 },
26318
910fa739f4a4 Fix PEP 293 related problems with --disable-unicode builds
Walter Dörwald <walter@livinglogic.de>
parents: 25441
diff changeset
788 #ifdef Py_USING_UNICODE
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
789 {
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
790 "ignore",
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
791 {
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
792 "ignore_errors",
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
793 ignore_errors,
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
794 METH_O,
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
795 PyDoc_STR("Implements the 'ignore' error handling, which "
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
796 "ignores malformed data and continues.")
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
797 }
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
798 },
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
799 {
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
800 "replace",
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
801 {
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
802 "replace_errors",
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
803 replace_errors,
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
804 METH_O,
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
805 PyDoc_STR("Implements the 'replace' error handling, which "
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
806 "replaces malformed data with a replacement marker.")
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
807 }
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
808 },
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
809 {
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
810 "xmlcharrefreplace",
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
811 {
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
812 "xmlcharrefreplace_errors",
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
813 xmlcharrefreplace_errors,
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
814 METH_O,
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
815 PyDoc_STR("Implements the 'xmlcharrefreplace' error handling, "
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
816 "which replaces an unencodable character with the "
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
817 "appropriate XML character reference.")
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
818 }
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
819 },
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
820 {
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
821 "backslashreplace",
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
822 {
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
823 "backslashreplace_errors",
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
824 backslashreplace_errors,
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
825 METH_O,
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
826 PyDoc_STR("Implements the 'backslashreplace' error handling, "
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
827 "which replaces an unencodable character with a "
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
828 "backslashed escape sequence.")
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
829 }
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
830 }
26318
910fa739f4a4 Fix PEP 293 related problems with --disable-unicode builds
Walter Dörwald <walter@livinglogic.de>
parents: 25441
diff changeset
831 #endif
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
832 };
28325
850626d47574 Fixed SF bug #663074. The codec system was using global static
Gustavo Niemeyer <gustavo@niemeyer.net>
parents: 27898
diff changeset
833
31797
846b3ee7e106 Changed random calls to PyThreadState_Get() to use the macro
Nicholas Bastin <nick.bastin@gmail.com>
parents: 31273
diff changeset
834 PyInterpreterState *interp = PyThreadState_GET()->interp;
28325
850626d47574 Fixed SF bug #663074. The codec system was using global static
Gustavo Niemeyer <gustavo@niemeyer.net>
parents: 27898
diff changeset
835 PyObject *mod;
32533
201986d6d9d6 Fix a couple of signed/unsigned comparison warnings
Neal Norwitz <nnorwitz@gmail.com>
parents: 31797
diff changeset
836 unsigned i;
28325
850626d47574 Fixed SF bug #663074. The codec system was using global static
Gustavo Niemeyer <gustavo@niemeyer.net>
parents: 27898
diff changeset
837
850626d47574 Fixed SF bug #663074. The codec system was using global static
Gustavo Niemeyer <gustavo@niemeyer.net>
parents: 27898
diff changeset
838 if (interp->codec_search_path != NULL)
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
839 return 0;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
840
28325
850626d47574 Fixed SF bug #663074. The codec system was using global static
Gustavo Niemeyer <gustavo@niemeyer.net>
parents: 27898
diff changeset
841 interp->codec_search_path = PyList_New(0);
850626d47574 Fixed SF bug #663074. The codec system was using global static
Gustavo Niemeyer <gustavo@niemeyer.net>
parents: 27898
diff changeset
842 interp->codec_search_cache = PyDict_New();
850626d47574 Fixed SF bug #663074. The codec system was using global static
Gustavo Niemeyer <gustavo@niemeyer.net>
parents: 27898
diff changeset
843 interp->codec_error_registry = PyDict_New();
850626d47574 Fixed SF bug #663074. The codec system was using global static
Gustavo Niemeyer <gustavo@niemeyer.net>
parents: 27898
diff changeset
844
850626d47574 Fixed SF bug #663074. The codec system was using global static
Gustavo Niemeyer <gustavo@niemeyer.net>
parents: 27898
diff changeset
845 if (interp->codec_error_registry) {
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
846 for (i = 0; i < sizeof(methods)/sizeof(methods[0]); ++i) {
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
847 PyObject *func = PyCFunction_New(&methods[i].def, NULL);
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
848 int res;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
849 if (!func)
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
850 Py_FatalError("can't initialize codec error registry");
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
851 res = PyCodec_RegisterError(methods[i].name, func);
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
852 Py_DECREF(func);
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
853 if (res)
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
854 Py_FatalError("can't initialize codec error registry");
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
855 }
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
856 }
28325
850626d47574 Fixed SF bug #663074. The codec system was using global static
Gustavo Niemeyer <gustavo@niemeyer.net>
parents: 27898
diff changeset
857
850626d47574 Fixed SF bug #663074. The codec system was using global static
Gustavo Niemeyer <gustavo@niemeyer.net>
parents: 27898
diff changeset
858 if (interp->codec_search_path == NULL ||
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
859 interp->codec_search_cache == NULL ||
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
860 interp->codec_error_registry == NULL)
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
861 Py_FatalError("can't initialize codec registry");
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
862
36712
5c36c222e7ca SF patch #1438387, PEP 328: relative and absolute imports.
Thomas Wouters <thomas@python.org>
parents: 36501
diff changeset
863 mod = PyImport_ImportModuleLevel("encodings", NULL, NULL, NULL, 0);
28325
850626d47574 Fixed SF bug #663074. The codec system was using global static
Gustavo Niemeyer <gustavo@niemeyer.net>
parents: 27898
diff changeset
864 if (mod == NULL) {
61463
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
865 if (PyErr_ExceptionMatches(PyExc_ImportError)) {
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
866 /* Ignore ImportErrors... this is done so that
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
867 distributions can disable the encodings package. Note
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
868 that other errors are not masked, e.g. SystemErrors
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
869 raised to inform the user of an error in the Python
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
870 configuration are still reported back to the user. */
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
871 PyErr_Clear();
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
872 return 0;
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
873 }
fd25030223b0 Untabify C files. Will watch buildbots.
Antoine Pitrou <solipsis@pitrou.net>
parents: 56824
diff changeset
874 return -1;
28325
850626d47574 Fixed SF bug #663074. The codec system was using global static
Gustavo Niemeyer <gustavo@niemeyer.net>
parents: 27898
diff changeset
875 }
850626d47574 Fixed SF bug #663074. The codec system was using global static
Gustavo Niemeyer <gustavo@niemeyer.net>
parents: 27898
diff changeset
876 Py_DECREF(mod);
850626d47574 Fixed SF bug #663074. The codec system was using global static
Gustavo Niemeyer <gustavo@niemeyer.net>
parents: 27898
diff changeset
877 return 0;
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
878 }