Mercurial > cpython-lru_cache-weakref

annotate Python/codecs.c @ 78856:e72cda3f40a2 3.2

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Issue #13992: The trashcan mechanism is now thread-safe. This eliminates sporadic crashes in multi-thread programs when several long deallocator chains ran concurrently and involved subclasses of built-in container types. Because of this change, a couple extension modules compiled for 3.2.4 (those which use the trashcan mechanism, despite it being undocumented) will not be loadable by 3.2.3 and earlier. However, extension modules compiled for 3.2.3 and earlier will be loadable by 3.2.4.
author Antoine Pitrou <solipsis@pitrou.net>
date Thu, 06 Sep 2012 00:59:49 +0200
parents 46b245f03f54
children 74d65c746f63 f55529aa023d
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
1 /* ------------------------------------------------------------------------
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
2
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
3 Python Codec Registry and support functions
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
4
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
5 Written by Marc-Andre Lemburg (mal@lemburg.com).
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
6
13828
254f8b7c4a83 Changing the CNRI copyright notice according to CNRI's instructions.
Guido van Rossum <guido@python.org>
parents: 13658
diff changeset
7 Copyright (c) Corporation for National Research Initiatives.
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
8
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
9 ------------------------------------------------------------------------ */
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
10
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
11 #include "Python.h"
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
12 #include <ctype.h>
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
13
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
14 /* --- Codec Registry ----------------------------------------------------- */
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
15
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
16 /* Import the standard encodings package which will register the first
43777
0d462d789b18 Merging the py3k-pep3137 branch back into the py3k branch.
Guido van Rossum <guido@python.org>
parents: 43603
diff changeset
17 codec search function.
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
18
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
19 This is done in a lazy way so that the Unicode implementation does
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
20 not downgrade startup time of scripts not needing it.
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
21
12195
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup
Guido van Rossum <guido@python.org>
parents: 12048
diff changeset
22 ImportErrors are silently ignored by this function. Only one try is
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup
Guido van Rossum <guido@python.org>
parents: 12048
diff changeset
23 made.
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
24
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
25 */
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
26
28325
850626d47574 Fixed SF bug #663074. The codec system was using global static
Gustavo Niemeyer <gustavo@niemeyer.net>
parents: 27898
diff changeset
27 static int _PyCodecRegistry_Init(void); /* Forward */
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
28
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
29 int PyCodec_Register(PyObject *search_function)
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
30 {
31797
846b3ee7e106 Changed random calls to PyThreadState_Get() to use the macro
Nicholas Bastin <nick.bastin@gmail.com>
parents: 31273
diff changeset
31 PyInterpreterState *interp = PyThreadState_GET()->interp;
28325
850626d47574 Fixed SF bug #663074. The codec system was using global static
Gustavo Niemeyer <gustavo@niemeyer.net>
parents: 27898
diff changeset
32 if (interp->codec_search_path == NULL && _PyCodecRegistry_Init())
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
33 goto onError;
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
34 if (search_function == NULL) {
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
35 PyErr_BadArgument();
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
36 goto onError;
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
37 }
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
38 if (!PyCallable_Check(search_function)) {
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
39 PyErr_SetString(PyExc_TypeError, "argument must be callable");
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
40 goto onError;
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
41 }
28325
850626d47574 Fixed SF bug #663074. The codec system was using global static
Gustavo Niemeyer <gustavo@niemeyer.net>
parents: 27898
diff changeset
42 return PyList_Append(interp->codec_search_path, search_function);
12195
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup
Guido van Rossum <guido@python.org>
parents: 12048
diff changeset
43
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup
Guido van Rossum <guido@python.org>
parents: 12048
diff changeset
44 onError:
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup
Guido van Rossum <guido@python.org>
parents: 12048
diff changeset
45 return -1;
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
46 }
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
47
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12195
diff changeset
48 /* Convert a string to a normalized Python string: all characters are
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12195
diff changeset
49 converted to lower case, spaces are replaced with underscores. */
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12195
diff changeset
50
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
51 static
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12195
diff changeset
52 PyObject *normalizestring(const char *string)
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
53 {
13128
cb928d926f27 Change the loop index in normalizestring() to size_t too, to avoid a
Guido van Rossum <guido@python.org>
parents: 13096
diff changeset
54 register size_t i;
13096
8c8de5b736dc Trent Mick's Win64 changes: size_t vs. int or long; also some overflow
Guido van Rossum <guido@python.org>
parents: 12648
diff changeset
55 size_t len = strlen(string);
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
56 char *p;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
57 PyObject *v;
43603
dd03aa741f56 This is the uncontroversial half of patch 1263 by Thomas Lee:
Guido van Rossum <guido@python.org>
parents: 42534
diff changeset
58
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37875
diff changeset
59 if (len > PY_SSIZE_T_MAX) {
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
60 PyErr_SetString(PyExc_OverflowError, "string is too large");
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
61 return NULL;
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37875
diff changeset
62 }
43603
dd03aa741f56 This is the uncontroversial half of patch 1263 by Thomas Lee:
Guido van Rossum <guido@python.org>
parents: 42534
diff changeset
63
dd03aa741f56 This is the uncontroversial half of patch 1263 by Thomas Lee:
Guido van Rossum <guido@python.org>
parents: 42534
diff changeset
64 p = PyMem_Malloc(len + 1);
dd03aa741f56 This is the uncontroversial half of patch 1263 by Thomas Lee:
Guido van Rossum <guido@python.org>
parents: 42534
diff changeset
65 if (p == NULL)
dd03aa741f56 This is the uncontroversial half of patch 1263 by Thomas Lee:
Guido van Rossum <guido@python.org>
parents: 42534
diff changeset
66 return NULL;
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12195
diff changeset
67 for (i = 0; i < len; i++) {
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12195
diff changeset
68 register char ch = string[i];
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12195
diff changeset
69 if (ch == ' ')
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12195
diff changeset
70 ch = '-';
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12195
diff changeset
71 else
71489
92d02de91cc9 Issue #1813: Fix codec lookup under Turkish locales.
Antoine Pitrou <solipsis@pitrou.net>
parents: 64590
diff changeset
72 ch = Py_TOLOWER(Py_CHARMASK(ch));
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
73 p[i] = ch;
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12195
diff changeset
74 }
43603
dd03aa741f56 This is the uncontroversial half of patch 1263 by Thomas Lee:
Guido van Rossum <guido@python.org>
parents: 42534
diff changeset
75 p[i] = '\0';
dd03aa741f56 This is the uncontroversial half of patch 1263 by Thomas Lee:
Guido van Rossum <guido@python.org>
parents: 42534
diff changeset
76 v = PyUnicode_FromString(p);
dd03aa741f56 This is the uncontroversial half of patch 1263 by Thomas Lee:
Guido van Rossum <guido@python.org>
parents: 42534
diff changeset
77 if (v == NULL)
dd03aa741f56 This is the uncontroversial half of patch 1263 by Thomas Lee:
Guido van Rossum <guido@python.org>
parents: 42534
diff changeset
78 return NULL;
dd03aa741f56 This is the uncontroversial half of patch 1263 by Thomas Lee:
Guido van Rossum <guido@python.org>
parents: 42534
diff changeset
79 PyMem_Free(p);
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
80 return v;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
81 }
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
82
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
83 /* Lookup the given encoding and return a tuple providing the codec
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
84 facilities.
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
85
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
86 The encoding string is looked up converted to all lower-case
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
87 characters. This makes encodings looked up through this mechanism
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
88 effectively case-insensitive.
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
89
43777
0d462d789b18 Merging the py3k-pep3137 branch back into the py3k branch.
Guido van Rossum <guido@python.org>
parents: 43603
diff changeset
90 If no codec is found, a LookupError is set and NULL returned.
12195
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup
Guido van Rossum <guido@python.org>
parents: 12048
diff changeset
91
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup
Guido van Rossum <guido@python.org>
parents: 12048
diff changeset
92 As side effect, this tries to load the encodings package, if not
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup
Guido van Rossum <guido@python.org>
parents: 12048
diff changeset
93 yet done. This is part of the lazy load strategy for the encodings
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup
Guido van Rossum <guido@python.org>
parents: 12048
diff changeset
94 package.
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup
Guido van Rossum <guido@python.org>
parents: 12048
diff changeset
95
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup
Guido van Rossum <guido@python.org>
parents: 12048
diff changeset
96 */
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
97
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
98 PyObject *_PyCodec_Lookup(const char *encoding)
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
99 {
28325
850626d47574 Fixed SF bug #663074. The codec system was using global static
Gustavo Niemeyer <gustavo@niemeyer.net>
parents: 27898
diff changeset
100 PyInterpreterState *interp;
12048
f87b56eebacc Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
101 PyObject *result, *args = NULL, *v;
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
102 Py_ssize_t i, len;
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
103
12648
2a53f5dc0590 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12248
diff changeset
104 if (encoding == NULL) {
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
105 PyErr_BadArgument();
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
106 goto onError;
12648
2a53f5dc0590 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12248
diff changeset
107 }
28325
850626d47574 Fixed SF bug #663074. The codec system was using global static
Gustavo Niemeyer <gustavo@niemeyer.net>
parents: 27898
diff changeset
108
31797
846b3ee7e106 Changed random calls to PyThreadState_Get() to use the macro
Nicholas Bastin <nick.bastin@gmail.com>
parents: 31273
diff changeset
109 interp = PyThreadState_GET()->interp;
28325
850626d47574 Fixed SF bug #663074. The codec system was using global static
Gustavo Niemeyer <gustavo@niemeyer.net>
parents: 27898
diff changeset
110 if (interp->codec_search_path == NULL && _PyCodecRegistry_Init())
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
111 goto onError;
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
112
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12195
diff changeset
113 /* Convert the encoding to a normalized Python string: all
13620
5b727ab37092 Spelling fixes supplied by Rob W. W. Hooft. All these are fixes in either
Thomas Wouters <thomas@python.org>
parents: 13330
diff changeset
114 characters are converted to lower case, spaces and hyphens are
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12195
diff changeset
115 replaced with underscores. */
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12195
diff changeset
116 v = normalizestring(encoding);
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
117 if (v == NULL)
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
118 goto onError;
43603
dd03aa741f56 This is the uncontroversial half of patch 1263 by Thomas Lee:
Guido van Rossum <guido@python.org>
parents: 42534
diff changeset
119 PyUnicode_InternInPlace(&v);
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
120
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
121 /* First, try to lookup the name in the registry dictionary */
28325
850626d47574 Fixed SF bug #663074. The codec system was using global static
Gustavo Niemeyer <gustavo@niemeyer.net>
parents: 27898
diff changeset
122 result = PyDict_GetItem(interp->codec_search_cache, v);
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
123 if (result != NULL) {
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
124 Py_INCREF(result);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
125 Py_DECREF(v);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
126 return result;
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
127 }
43777
0d462d789b18 Merging the py3k-pep3137 branch back into the py3k branch.
Guido van Rossum <guido@python.org>
parents: 43603
diff changeset
128
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
129 /* Next, scan the search functions in order of registration */
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
130 args = PyTuple_New(1);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
131 if (args == NULL)
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
132 goto onError;
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
133 PyTuple_SET_ITEM(args,0,v);
12048
f87b56eebacc Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
134
28325
850626d47574 Fixed SF bug #663074. The codec system was using global static
Gustavo Niemeyer <gustavo@niemeyer.net>
parents: 27898
diff changeset
135 len = PyList_Size(interp->codec_search_path);
12048
f87b56eebacc Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
136 if (len < 0)
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
137 goto onError;
12195
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup
Guido van Rossum <guido@python.org>
parents: 12048
diff changeset
138 if (len == 0) {
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
139 PyErr_SetString(PyExc_LookupError,
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
140 "no codec search functions registered: "
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
141 "can't find encoding");
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
142 goto onError;
12195
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup
Guido van Rossum <guido@python.org>
parents: 12048
diff changeset
143 }
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
144
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
145 for (i = 0; i < len; i++) {
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
146 PyObject *func;
43777
0d462d789b18 Merging the py3k-pep3137 branch back into the py3k branch.
Guido van Rossum <guido@python.org>
parents: 43603
diff changeset
147
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
148 func = PyList_GetItem(interp->codec_search_path, i);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
149 if (func == NULL)
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
150 goto onError;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
151 result = PyEval_CallObject(func, args);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
152 if (result == NULL)
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
153 goto onError;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
154 if (result == Py_None) {
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
155 Py_DECREF(result);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
156 continue;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
157 }
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
158 if (!PyTuple_Check(result) || PyTuple_GET_SIZE(result) != 4) {
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
159 PyErr_SetString(PyExc_TypeError,
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
160 "codec search functions must return 4-tuples");
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
161 Py_DECREF(result);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
162 goto onError;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
163 }
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
164 break;
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
165 }
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
166 if (i == len) {
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
167 /* XXX Perhaps we should cache misses too ? */
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
168 PyErr_Format(PyExc_LookupError,
25441
d32588c88633 Add encoding name in LookupError. Fixes #615013. Will backport to 2.2.
Martin v. Löwis <martin@v.loewis.de>
parents: 25095
diff changeset
169 "unknown encoding: %s", encoding);
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
170 goto onError;
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
171 }
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
172
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
173 /* Cache and return the result */
42534
5f9e79d1e158 Handle error
Neal Norwitz <nnorwitz@gmail.com>
parents: 42057
diff changeset
174 if (PyDict_SetItem(interp->codec_search_cache, v, result) < 0) {
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
175 Py_DECREF(result);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
176 goto onError;
42534
5f9e79d1e158 Handle error
Neal Norwitz <nnorwitz@gmail.com>
parents: 42057
diff changeset
177 }
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
178 Py_DECREF(args);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
179 return result;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
180
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
181 onError:
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
182 Py_XDECREF(args);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
183 return NULL;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
184 }
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
185
47496
2284fa89ab08 Move the codec decode type checks to bytes/bytearray.decode().
Marc-André Lemburg <mal@egenix.com>
parents: 47301
diff changeset
186 /* Codec registry encoding check API. */
2284fa89ab08 Move the codec decode type checks to bytes/bytearray.decode().
Marc-André Lemburg <mal@egenix.com>
parents: 47301
diff changeset
187
2284fa89ab08 Move the codec decode type checks to bytes/bytearray.decode().
Marc-André Lemburg <mal@egenix.com>
parents: 47301
diff changeset
188 int PyCodec_KnownEncoding(const char *encoding)
2284fa89ab08 Move the codec decode type checks to bytes/bytearray.decode().
Marc-André Lemburg <mal@egenix.com>
parents: 47301
diff changeset
189 {
2284fa89ab08 Move the codec decode type checks to bytes/bytearray.decode().
Marc-André Lemburg <mal@egenix.com>
parents: 47301
diff changeset
190 PyObject *codecs;
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
191
47496
2284fa89ab08 Move the codec decode type checks to bytes/bytearray.decode().
Marc-André Lemburg <mal@egenix.com>
parents: 47301
diff changeset
192 codecs = _PyCodec_Lookup(encoding);
2284fa89ab08 Move the codec decode type checks to bytes/bytearray.decode().
Marc-André Lemburg <mal@egenix.com>
parents: 47301
diff changeset
193 if (!codecs) {
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
194 PyErr_Clear();
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
195 return 0;
47496
2284fa89ab08 Move the codec decode type checks to bytes/bytearray.decode().
Marc-André Lemburg <mal@egenix.com>
parents: 47301
diff changeset
196 }
2284fa89ab08 Move the codec decode type checks to bytes/bytearray.decode().
Marc-André Lemburg <mal@egenix.com>
parents: 47301
diff changeset
197 else {
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
198 Py_DECREF(codecs);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
199 return 1;
47496
2284fa89ab08 Move the codec decode type checks to bytes/bytearray.decode().
Marc-André Lemburg <mal@egenix.com>
parents: 47301
diff changeset
200 }
2284fa89ab08 Move the codec decode type checks to bytes/bytearray.decode().
Marc-André Lemburg <mal@egenix.com>
parents: 47301
diff changeset
201 }
2284fa89ab08 Move the codec decode type checks to bytes/bytearray.decode().
Marc-André Lemburg <mal@egenix.com>
parents: 47301
diff changeset
202
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
203 static
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
204 PyObject *args_tuple(PyObject *object,
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
205 const char *errors)
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
206 {
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
207 PyObject *args;
43777
0d462d789b18 Merging the py3k-pep3137 branch back into the py3k branch.
Guido van Rossum <guido@python.org>
parents: 43603
diff changeset
208
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
209 args = PyTuple_New(1 + (errors != NULL));
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
210 if (args == NULL)
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
211 return NULL;
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
212 Py_INCREF(object);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
213 PyTuple_SET_ITEM(args,0,object);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
214 if (errors) {
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
215 PyObject *v;
43777
0d462d789b18 Merging the py3k-pep3137 branch back into the py3k branch.
Guido van Rossum <guido@python.org>
parents: 43603
diff changeset
216
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
217 v = PyUnicode_FromString(errors);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
218 if (v == NULL) {
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
219 Py_DECREF(args);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
220 return NULL;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
221 }
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
222 PyTuple_SET_ITEM(args, 1, v);
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
223 }
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
224 return args;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
225 }
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
226
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37875
diff changeset
227 /* Helper function to get a codec item */
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37875
diff changeset
228
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37875
diff changeset
229 static
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37875
diff changeset
230 PyObject *codec_getitem(const char *encoding, int index)
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37875
diff changeset
231 {
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37875
diff changeset
232 PyObject *codecs;
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37875
diff changeset
233 PyObject *v;
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37875
diff changeset
234
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37875
diff changeset
235 codecs = _PyCodec_Lookup(encoding);
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37875
diff changeset
236 if (codecs == NULL)
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
237 return NULL;
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37875
diff changeset
238 v = PyTuple_GET_ITEM(codecs, index);
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37875
diff changeset
239 Py_DECREF(codecs);
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37875
diff changeset
240 Py_INCREF(v);
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37875
diff changeset
241 return v;
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37875
diff changeset
242 }
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37875
diff changeset
243
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37875
diff changeset
244 /* Helper function to create an incremental codec. */
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
245
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
246 static
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37875
diff changeset
247 PyObject *codec_getincrementalcodec(const char *encoding,
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
248 const char *errors,
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
249 const char *attrname)
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
250 {
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37875
diff changeset
251 PyObject *codecs, *ret, *inccodec;
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
252
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37875
diff changeset
253 codecs = _PyCodec_Lookup(encoding);
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37875
diff changeset
254 if (codecs == NULL)
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
255 return NULL;
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37875
diff changeset
256 inccodec = PyObject_GetAttrString(codecs, attrname);
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37875
diff changeset
257 Py_DECREF(codecs);
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37875
diff changeset
258 if (inccodec == NULL)
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
259 return NULL;
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37875
diff changeset
260 if (errors)
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
261 ret = PyObject_CallFunction(inccodec, "s", errors);
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37875
diff changeset
262 else
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
263 ret = PyObject_CallFunction(inccodec, NULL);
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37875
diff changeset
264 Py_DECREF(inccodec);
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37875
diff changeset
265 return ret;
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37875
diff changeset
266 }
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37875
diff changeset
267
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37875
diff changeset
268 /* Helper function to create a stream codec. */
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37875
diff changeset
269
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37875
diff changeset
270 static
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37875
diff changeset
271 PyObject *codec_getstreamcodec(const char *encoding,
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
272 PyObject *stream,
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
273 const char *errors,
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
274 const int index)
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37875
diff changeset
275 {
39442
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
276 PyObject *codecs, *streamcodec, *codeccls;
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37875
diff changeset
277
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37875
diff changeset
278 codecs = _PyCodec_Lookup(encoding);
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37875
diff changeset
279 if (codecs == NULL)
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
280 return NULL;
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37875
diff changeset
281
39442
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
282 codeccls = PyTuple_GET_ITEM(codecs, index);
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
283 if (errors != NULL)
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
284 streamcodec = PyObject_CallFunction(codeccls, "Os", stream, errors);
39442
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
285 else
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
286 streamcodec = PyObject_CallFunction(codeccls, "O", stream);
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37875
diff changeset
287 Py_DECREF(codecs);
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37875
diff changeset
288 return streamcodec;
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
289 }
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
290
43777
0d462d789b18 Merging the py3k-pep3137 branch back into the py3k branch.
Guido van Rossum <guido@python.org>
parents: 43603
diff changeset
291 /* Convenience APIs to query the Codec registry.
0d462d789b18 Merging the py3k-pep3137 branch back into the py3k branch.
Guido van Rossum <guido@python.org>
parents: 43603
diff changeset
292
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
293 All APIs return a codec object with incremented refcount.
43777
0d462d789b18 Merging the py3k-pep3137 branch back into the py3k branch.
Guido van Rossum <guido@python.org>
parents: 43603
diff changeset
294
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
295 */
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
296
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
297 PyObject *PyCodec_Encoder(const char *encoding)
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
298 {
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37875
diff changeset
299 return codec_getitem(encoding, 0);
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
300 }
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
301
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
302 PyObject *PyCodec_Decoder(const char *encoding)
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
303 {
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37875
diff changeset
304 return codec_getitem(encoding, 1);
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
305 }
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
306
37875
a4c70f3968be Merge part of the trunk changes into the p3yk branch. This merges from 43030
Thomas Wouters <thomas@python.org>
parents: 36751
diff changeset
307 PyObject *PyCodec_IncrementalEncoder(const char *encoding,
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
308 const char *errors)
37875
a4c70f3968be Merge part of the trunk changes into the p3yk branch. This merges from 43030
Thomas Wouters <thomas@python.org>
parents: 36751
diff changeset
309 {
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37875
diff changeset
310 return codec_getincrementalcodec(encoding, errors, "incrementalencoder");
37875
a4c70f3968be Merge part of the trunk changes into the p3yk branch. This merges from 43030
Thomas Wouters <thomas@python.org>
parents: 36751
diff changeset
311 }
a4c70f3968be Merge part of the trunk changes into the p3yk branch. This merges from 43030
Thomas Wouters <thomas@python.org>
parents: 36751
diff changeset
312
a4c70f3968be Merge part of the trunk changes into the p3yk branch. This merges from 43030
Thomas Wouters <thomas@python.org>
parents: 36751
diff changeset
313 PyObject *PyCodec_IncrementalDecoder(const char *encoding,
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
314 const char *errors)
37875
a4c70f3968be Merge part of the trunk changes into the p3yk branch. This merges from 43030
Thomas Wouters <thomas@python.org>
parents: 36751
diff changeset
315 {
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37875
diff changeset
316 return codec_getincrementalcodec(encoding, errors, "incrementaldecoder");
37875
a4c70f3968be Merge part of the trunk changes into the p3yk branch. This merges from 43030
Thomas Wouters <thomas@python.org>
parents: 36751
diff changeset
317 }
a4c70f3968be Merge part of the trunk changes into the p3yk branch. This merges from 43030
Thomas Wouters <thomas@python.org>
parents: 36751
diff changeset
318
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
319 PyObject *PyCodec_StreamReader(const char *encoding,
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
320 PyObject *stream,
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
321 const char *errors)
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
322 {
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37875
diff changeset
323 return codec_getstreamcodec(encoding, stream, errors, 2);
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
324 }
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
325
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
326 PyObject *PyCodec_StreamWriter(const char *encoding,
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
327 PyObject *stream,
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
328 const char *errors)
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
329 {
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37875
diff changeset
330 return codec_getstreamcodec(encoding, stream, errors, 3);
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
331 }
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
332
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
333 /* Encode an object (e.g. an Unicode object) using the given encoding
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
334 and return the resulting encoded object (usually a Python string).
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
335
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
336 errors is passed to the encoder factory as argument if non-NULL. */
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
337
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
338 PyObject *PyCodec_Encode(PyObject *object,
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
339 const char *encoding,
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
340 const char *errors)
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
341 {
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
342 PyObject *encoder = NULL;
35939
79374eb6db51 Fix a few more ref leaks. Backport candidate
Neal Norwitz <nnorwitz@gmail.com>
parents: 32533
diff changeset
343 PyObject *args = NULL, *result = NULL;
43777
0d462d789b18 Merging the py3k-pep3137 branch back into the py3k branch.
Guido van Rossum <guido@python.org>
parents: 43603
diff changeset
344 PyObject *v = NULL;
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
345
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
346 encoder = PyCodec_Encoder(encoding);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
347 if (encoder == NULL)
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
348 goto onError;
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
349
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
350 args = args_tuple(object, errors);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
351 if (args == NULL)
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
352 goto onError;
43777
0d462d789b18 Merging the py3k-pep3137 branch back into the py3k branch.
Guido van Rossum <guido@python.org>
parents: 43603
diff changeset
353
0d462d789b18 Merging the py3k-pep3137 branch back into the py3k branch.
Guido van Rossum <guido@python.org>
parents: 43603
diff changeset
354 result = PyEval_CallObject(encoder, args);
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
355 if (result == NULL)
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
356 goto onError;
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
357
43777
0d462d789b18 Merging the py3k-pep3137 branch back into the py3k branch.
Guido van Rossum <guido@python.org>
parents: 43603
diff changeset
358 if (!PyTuple_Check(result) ||
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
359 PyTuple_GET_SIZE(result) != 2) {
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
360 PyErr_SetString(PyExc_TypeError,
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
361 "encoder must return a tuple (object, integer)");
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
362 goto onError;
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
363 }
47496
2284fa89ab08 Move the codec decode type checks to bytes/bytearray.decode().
Marc-André Lemburg <mal@egenix.com>
parents: 47301
diff changeset
364 v = PyTuple_GET_ITEM(result,0);
2284fa89ab08 Move the codec decode type checks to bytes/bytearray.decode().
Marc-André Lemburg <mal@egenix.com>
parents: 47301
diff changeset
365 Py_INCREF(v);
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
366 /* We don't check or use the second (integer) entry. */
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
367
47496
2284fa89ab08 Move the codec decode type checks to bytes/bytearray.decode().
Marc-André Lemburg <mal@egenix.com>
parents: 47301
diff changeset
368 Py_DECREF(args);
2284fa89ab08 Move the codec decode type checks to bytes/bytearray.decode().
Marc-André Lemburg <mal@egenix.com>
parents: 47301
diff changeset
369 Py_DECREF(encoder);
2284fa89ab08 Move the codec decode type checks to bytes/bytearray.decode().
Marc-André Lemburg <mal@egenix.com>
parents: 47301
diff changeset
370 Py_DECREF(result);
2284fa89ab08 Move the codec decode type checks to bytes/bytearray.decode().
Marc-André Lemburg <mal@egenix.com>
parents: 47301
diff changeset
371 return v;
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
372
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
373 onError:
35939
79374eb6db51 Fix a few more ref leaks. Backport candidate
Neal Norwitz <nnorwitz@gmail.com>
parents: 32533
diff changeset
374 Py_XDECREF(result);
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
375 Py_XDECREF(args);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
376 Py_XDECREF(encoder);
47496
2284fa89ab08 Move the codec decode type checks to bytes/bytearray.decode().
Marc-André Lemburg <mal@egenix.com>
parents: 47301
diff changeset
377 return NULL;
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
378 }
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
379
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
380 /* Decode an object (usually a Python string) using the given encoding
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
381 and return an equivalent object (e.g. an Unicode object).
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
382
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
383 errors is passed to the decoder factory as argument if non-NULL. */
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
384
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
385 PyObject *PyCodec_Decode(PyObject *object,
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
386 const char *encoding,
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
387 const char *errors)
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
388 {
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
389 PyObject *decoder = NULL;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
390 PyObject *args = NULL, *result = NULL;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
391 PyObject *v;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
392
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
393 decoder = PyCodec_Decoder(encoding);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
394 if (decoder == NULL)
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
395 goto onError;
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
396
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
397 args = args_tuple(object, errors);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
398 if (args == NULL)
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
399 goto onError;
43777
0d462d789b18 Merging the py3k-pep3137 branch back into the py3k branch.
Guido van Rossum <guido@python.org>
parents: 43603
diff changeset
400
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
401 result = PyEval_CallObject(decoder,args);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
402 if (result == NULL)
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
403 goto onError;
43777
0d462d789b18 Merging the py3k-pep3137 branch back into the py3k branch.
Guido van Rossum <guido@python.org>
parents: 43603
diff changeset
404 if (!PyTuple_Check(result) ||
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
405 PyTuple_GET_SIZE(result) != 2) {
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
406 PyErr_SetString(PyExc_TypeError,
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
407 "decoder must return a tuple (object,integer)");
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
408 goto onError;
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
409 }
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
410 v = PyTuple_GET_ITEM(result,0);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
411 Py_INCREF(v);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
412 /* We don't check or use the second (integer) entry. */
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
413
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
414 Py_DECREF(args);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
415 Py_DECREF(decoder);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
416 Py_DECREF(result);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
417 return v;
43777
0d462d789b18 Merging the py3k-pep3137 branch back into the py3k branch.
Guido van Rossum <guido@python.org>
parents: 43603
diff changeset
418
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
419 onError:
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
420 Py_XDECREF(args);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
421 Py_XDECREF(decoder);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
422 Py_XDECREF(result);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
423 return NULL;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
424 }
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
425
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
426 /* Register the error handling callback function error under the name
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
427 name. This function will be called by the codec when it encounters
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
428 an unencodable characters/undecodable bytes and doesn't know the
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
429 callback name, when name is specified as the error parameter
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
430 in the call to the encode/decode function.
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
431 Return 0 on success, -1 on error */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
432 int PyCodec_RegisterError(const char *name, PyObject *error)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
433 {
31797
846b3ee7e106 Changed random calls to PyThreadState_Get() to use the macro
Nicholas Bastin <nick.bastin@gmail.com>
parents: 31273
diff changeset
434 PyInterpreterState *interp = PyThreadState_GET()->interp;
28325
850626d47574 Fixed SF bug #663074. The codec system was using global static
Gustavo Niemeyer <gustavo@niemeyer.net>
parents: 27898
diff changeset
435 if (interp->codec_search_path == NULL && _PyCodecRegistry_Init())
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
436 return -1;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
437 if (!PyCallable_Check(error)) {
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
438 PyErr_SetString(PyExc_TypeError, "handler must be callable");
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
439 return -1;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
440 }
28325
850626d47574 Fixed SF bug #663074. The codec system was using global static
Gustavo Niemeyer <gustavo@niemeyer.net>
parents: 27898
diff changeset
441 return PyDict_SetItemString(interp->codec_error_registry,
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
442 (char *)name, error);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
443 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
444
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
445 /* Lookup the error handling callback function registered under the
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
446 name error. As a special case NULL can be passed, in which case
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
447 the error handling callback for strict encoding will be returned. */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
448 PyObject *PyCodec_LookupError(const char *name)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
449 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
450 PyObject *handler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
451
31797
846b3ee7e106 Changed random calls to PyThreadState_Get() to use the macro
Nicholas Bastin <nick.bastin@gmail.com>
parents: 31273
diff changeset
452 PyInterpreterState *interp = PyThreadState_GET()->interp;
28325
850626d47574 Fixed SF bug #663074. The codec system was using global static
Gustavo Niemeyer <gustavo@niemeyer.net>
parents: 27898
diff changeset
453 if (interp->codec_search_path == NULL && _PyCodecRegistry_Init())
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
454 return NULL;
28325
850626d47574 Fixed SF bug #663074. The codec system was using global static
Gustavo Niemeyer <gustavo@niemeyer.net>
parents: 27898
diff changeset
455
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
456 if (name==NULL)
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
457 name = "strict";
28325
850626d47574 Fixed SF bug #663074. The codec system was using global static
Gustavo Niemeyer <gustavo@niemeyer.net>
parents: 27898
diff changeset
458 handler = PyDict_GetItemString(interp->codec_error_registry, (char *)name);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
459 if (!handler)
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
460 PyErr_Format(PyExc_LookupError, "unknown error handler name '%.400s'", name);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
461 else
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
462 Py_INCREF(handler);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
463 return handler;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
464 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
465
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
466 static void wrong_exception_type(PyObject *exc)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
467 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
468 PyObject *type = PyObject_GetAttrString(exc, "__class__");
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
469 if (type != NULL) {
41878
089f449a20cd Change PyErr_Format() to generate a unicode string (by using
Walter Dörwald <walter@livinglogic.de>
parents: 41587
diff changeset
470 PyObject *name = PyObject_GetAttrString(type, "__name__");
089f449a20cd Change PyErr_Format() to generate a unicode string (by using
Walter Dörwald <walter@livinglogic.de>
parents: 41587
diff changeset
471 Py_DECREF(type);
089f449a20cd Change PyErr_Format() to generate a unicode string (by using
Walter Dörwald <walter@livinglogic.de>
parents: 41587
diff changeset
472 if (name != NULL) {
089f449a20cd Change PyErr_Format() to generate a unicode string (by using
Walter Dörwald <walter@livinglogic.de>
parents: 41587
diff changeset
473 PyErr_Format(PyExc_TypeError,
089f449a20cd Change PyErr_Format() to generate a unicode string (by using
Walter Dörwald <walter@livinglogic.de>
parents: 41587
diff changeset
474 "don't know how to handle %S in error callback", name);
089f449a20cd Change PyErr_Format() to generate a unicode string (by using
Walter Dörwald <walter@livinglogic.de>
parents: 41587
diff changeset
475 Py_DECREF(name);
089f449a20cd Change PyErr_Format() to generate a unicode string (by using
Walter Dörwald <walter@livinglogic.de>
parents: 41587
diff changeset
476 }
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
477 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
478 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
479
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
480 PyObject *PyCodec_StrictErrors(PyObject *exc)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
481 {
36751
8e04de2480e4 PEP 352 implementation. Creates a new base class, BaseException, which has an
Brett Cannon <bcannon@gmail.com>
parents: 36712
diff changeset
482 if (PyExceptionInstance_Check(exc))
8e04de2480e4 PEP 352 implementation. Creates a new base class, BaseException, which has an
Brett Cannon <bcannon@gmail.com>
parents: 36712
diff changeset
483 PyErr_SetObject(PyExceptionInstance_Class(exc), exc);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
484 else
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
485 PyErr_SetString(PyExc_TypeError, "codec must pass exception instance");
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
486 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
487 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
488
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
489
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
490 PyObject *PyCodec_IgnoreErrors(PyObject *exc)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
491 {
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 35939
diff changeset
492 Py_ssize_t end;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
493 if (PyObject_IsInstance(exc, PyExc_UnicodeEncodeError)) {
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
494 if (PyUnicodeEncodeError_GetEnd(exc, &end))
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
495 return NULL;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
496 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
497 else if (PyObject_IsInstance(exc, PyExc_UnicodeDecodeError)) {
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
498 if (PyUnicodeDecodeError_GetEnd(exc, &end))
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
499 return NULL;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
500 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
501 else if (PyObject_IsInstance(exc, PyExc_UnicodeTranslateError)) {
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
502 if (PyUnicodeTranslateError_GetEnd(exc, &end))
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
503 return NULL;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
504 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
505 else {
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
506 wrong_exception_type(exc);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
507 return NULL;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
508 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
509 /* ouch: passing NULL, 0, pos gives None instead of u'' */
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 35939
diff changeset
510 return Py_BuildValue("(u#n)", &end, 0, end);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
511 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
512
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
513
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
514 PyObject *PyCodec_ReplaceErrors(PyObject *exc)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
515 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
516 PyObject *restuple;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 35939
diff changeset
517 Py_ssize_t start;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 35939
diff changeset
518 Py_ssize_t end;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 35939
diff changeset
519 Py_ssize_t i;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
520
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
521 if (PyObject_IsInstance(exc, PyExc_UnicodeEncodeError)) {
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
522 PyObject *res;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
523 Py_UNICODE *p;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
524 if (PyUnicodeEncodeError_GetStart(exc, &start))
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
525 return NULL;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
526 if (PyUnicodeEncodeError_GetEnd(exc, &end))
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
527 return NULL;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
528 res = PyUnicode_FromUnicode(NULL, end-start);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
529 if (res == NULL)
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
530 return NULL;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
531 for (p = PyUnicode_AS_UNICODE(res), i = start;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
532 i<end; ++p, ++i)
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
533 *p = '?';
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
534 restuple = Py_BuildValue("(On)", res, end);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
535 Py_DECREF(res);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
536 return restuple;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
537 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
538 else if (PyObject_IsInstance(exc, PyExc_UnicodeDecodeError)) {
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
539 Py_UNICODE res = Py_UNICODE_REPLACEMENT_CHARACTER;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
540 if (PyUnicodeDecodeError_GetEnd(exc, &end))
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
541 return NULL;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
542 return Py_BuildValue("(u#n)", &res, 1, end);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
543 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
544 else if (PyObject_IsInstance(exc, PyExc_UnicodeTranslateError)) {
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
545 PyObject *res;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
546 Py_UNICODE *p;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
547 if (PyUnicodeTranslateError_GetStart(exc, &start))
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
548 return NULL;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
549 if (PyUnicodeTranslateError_GetEnd(exc, &end))
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
550 return NULL;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
551 res = PyUnicode_FromUnicode(NULL, end-start);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
552 if (res == NULL)
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
553 return NULL;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
554 for (p = PyUnicode_AS_UNICODE(res), i = start;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
555 i<end; ++p, ++i)
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
556 *p = Py_UNICODE_REPLACEMENT_CHARACTER;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
557 restuple = Py_BuildValue("(On)", res, end);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
558 Py_DECREF(res);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
559 return restuple;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
560 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
561 else {
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
562 wrong_exception_type(exc);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
563 return NULL;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
564 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
565 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
566
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
567 PyObject *PyCodec_XMLCharRefReplaceErrors(PyObject *exc)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
568 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
569 if (PyObject_IsInstance(exc, PyExc_UnicodeEncodeError)) {
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
570 PyObject *restuple;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
571 PyObject *object;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
572 Py_ssize_t start;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
573 Py_ssize_t end;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
574 PyObject *res;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
575 Py_UNICODE *p;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
576 Py_UNICODE *startp;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
577 Py_UNICODE *outp;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
578 int ressize;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
579 if (PyUnicodeEncodeError_GetStart(exc, &start))
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
580 return NULL;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
581 if (PyUnicodeEncodeError_GetEnd(exc, &end))
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
582 return NULL;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
583 if (!(object = PyUnicodeEncodeError_GetObject(exc)))
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
584 return NULL;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
585 startp = PyUnicode_AS_UNICODE(object);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
586 for (p = startp+start, ressize = 0; p < startp+end; ++p) {
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
587 if (*p<10)
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
588 ressize += 2+1+1;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
589 else if (*p<100)
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
590 ressize += 2+2+1;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
591 else if (*p<1000)
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
592 ressize += 2+3+1;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
593 else if (*p<10000)
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
594 ressize += 2+4+1;
31273
25519d6c97b8 Fix gcc 3.3 warnings related to Py_UNICODE_WIDE.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 28325
diff changeset
595 #ifndef Py_UNICODE_WIDE
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
596 else
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
597 ressize += 2+5+1;
31273
25519d6c97b8 Fix gcc 3.3 warnings related to Py_UNICODE_WIDE.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 28325
diff changeset
598 #else
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
599 else if (*p<100000)
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
600 ressize += 2+5+1;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
601 else if (*p<1000000)
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
602 ressize += 2+6+1;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
603 else
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
604 ressize += 2+7+1;
31273
25519d6c97b8 Fix gcc 3.3 warnings related to Py_UNICODE_WIDE.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 28325
diff changeset
605 #endif
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
606 }
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
607 /* allocate replacement */
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
608 res = PyUnicode_FromUnicode(NULL, ressize);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
609 if (res == NULL) {
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
610 Py_DECREF(object);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
611 return NULL;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
612 }
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
613 /* generate replacement */
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
614 for (p = startp+start, outp = PyUnicode_AS_UNICODE(res);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
615 p < startp+end; ++p) {
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
616 Py_UNICODE c = *p;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
617 int digits;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
618 int base;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
619 *outp++ = '&';
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
620 *outp++ = '#';
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
621 if (*p<10) {
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
622 digits = 1;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
623 base = 1;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
624 }
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
625 else if (*p<100) {
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
626 digits = 2;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
627 base = 10;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
628 }
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
629 else if (*p<1000) {
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
630 digits = 3;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
631 base = 100;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
632 }
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
633 else if (*p<10000) {
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
634 digits = 4;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
635 base = 1000;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
636 }
31273
25519d6c97b8 Fix gcc 3.3 warnings related to Py_UNICODE_WIDE.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 28325
diff changeset
637 #ifndef Py_UNICODE_WIDE
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
638 else {
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
639 digits = 5;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
640 base = 10000;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
641 }
31273
25519d6c97b8 Fix gcc 3.3 warnings related to Py_UNICODE_WIDE.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 28325
diff changeset
642 #else
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
643 else if (*p<100000) {
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
644 digits = 5;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
645 base = 10000;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
646 }
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
647 else if (*p<1000000) {
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
648 digits = 6;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
649 base = 100000;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
650 }
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
651 else {
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
652 digits = 7;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
653 base = 1000000;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
654 }
31273
25519d6c97b8 Fix gcc 3.3 warnings related to Py_UNICODE_WIDE.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 28325
diff changeset
655 #endif
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
656 while (digits-->0) {
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
657 *outp++ = '0' + c/base;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
658 c %= base;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
659 base /= 10;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
660 }
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
661 *outp++ = ';';
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
662 }
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
663 restuple = Py_BuildValue("(On)", res, end);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
664 Py_DECREF(res);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
665 Py_DECREF(object);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
666 return restuple;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
667 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
668 else {
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
669 wrong_exception_type(exc);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
670 return NULL;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
671 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
672 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
673
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
674 static Py_UNICODE hexdigits[] = {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
675 '0', '1', '2', '3', '4', '5', '6', '7',
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
676 '8', '9', 'a', 'b', 'c', 'd', 'e', 'f'
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
677 };
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
678
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
679 PyObject *PyCodec_BackslashReplaceErrors(PyObject *exc)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
680 {
64590
fc76e8c617dc Issue #9804: ascii() now always represents unicode surrogate pairs as
Antoine Pitrou <solipsis@pitrou.net>
parents: 61466
diff changeset
681 #ifndef Py_UNICODE_WIDE
fc76e8c617dc Issue #9804: ascii() now always represents unicode surrogate pairs as
Antoine Pitrou <solipsis@pitrou.net>
parents: 61466
diff changeset
682 #define IS_SURROGATE_PAIR(p, end) \
fc76e8c617dc Issue #9804: ascii() now always represents unicode surrogate pairs as
Antoine Pitrou <solipsis@pitrou.net>
parents: 61466
diff changeset
683 (*p >= 0xD800 && *p <= 0xDBFF && (p + 1) < end && \
fc76e8c617dc Issue #9804: ascii() now always represents unicode surrogate pairs as
Antoine Pitrou <solipsis@pitrou.net>
parents: 61466
diff changeset
684 *(p + 1) >= 0xDC00 && *(p + 1) <= 0xDFFF)
fc76e8c617dc Issue #9804: ascii() now always represents unicode surrogate pairs as
Antoine Pitrou <solipsis@pitrou.net>
parents: 61466
diff changeset
685 #else
fc76e8c617dc Issue #9804: ascii() now always represents unicode surrogate pairs as
Antoine Pitrou <solipsis@pitrou.net>
parents: 61466
diff changeset
686 #define IS_SURROGATE_PAIR(p, end) 0
fc76e8c617dc Issue #9804: ascii() now always represents unicode surrogate pairs as
Antoine Pitrou <solipsis@pitrou.net>
parents: 61466
diff changeset
687 #endif
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
688 if (PyObject_IsInstance(exc, PyExc_UnicodeEncodeError)) {
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
689 PyObject *restuple;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
690 PyObject *object;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
691 Py_ssize_t start;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
692 Py_ssize_t end;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
693 PyObject *res;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
694 Py_UNICODE *p;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
695 Py_UNICODE *startp;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
696 Py_UNICODE *outp;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
697 int ressize;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
698 if (PyUnicodeEncodeError_GetStart(exc, &start))
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
699 return NULL;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
700 if (PyUnicodeEncodeError_GetEnd(exc, &end))
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
701 return NULL;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
702 if (!(object = PyUnicodeEncodeError_GetObject(exc)))
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
703 return NULL;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
704 startp = PyUnicode_AS_UNICODE(object);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
705 for (p = startp+start, ressize = 0; p < startp+end; ++p) {
31273
25519d6c97b8 Fix gcc 3.3 warnings related to Py_UNICODE_WIDE.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 28325
diff changeset
706 #ifdef Py_UNICODE_WIDE
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
707 if (*p >= 0x00010000)
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
708 ressize += 1+1+8;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
709 else
31273
25519d6c97b8 Fix gcc 3.3 warnings related to Py_UNICODE_WIDE.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 28325
diff changeset
710 #endif
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
711 if (*p >= 0x100) {
64590
fc76e8c617dc Issue #9804: ascii() now always represents unicode surrogate pairs as
Antoine Pitrou <solipsis@pitrou.net>
parents: 61466
diff changeset
712 if (IS_SURROGATE_PAIR(p, startp+end)) {
fc76e8c617dc Issue #9804: ascii() now always represents unicode surrogate pairs as
Antoine Pitrou <solipsis@pitrou.net>
parents: 61466
diff changeset
713 ressize += 1+1+8;
fc76e8c617dc Issue #9804: ascii() now always represents unicode surrogate pairs as
Antoine Pitrou <solipsis@pitrou.net>
parents: 61466
diff changeset
714 ++p;
fc76e8c617dc Issue #9804: ascii() now always represents unicode surrogate pairs as
Antoine Pitrou <solipsis@pitrou.net>
parents: 61466
diff changeset
715 }
fc76e8c617dc Issue #9804: ascii() now always represents unicode surrogate pairs as
Antoine Pitrou <solipsis@pitrou.net>
parents: 61466
diff changeset
716 else
fc76e8c617dc Issue #9804: ascii() now always represents unicode surrogate pairs as
Antoine Pitrou <solipsis@pitrou.net>
parents: 61466
diff changeset
717 ressize += 1+1+4;
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
718 }
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
719 else
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
720 ressize += 1+1+2;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
721 }
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
722 res = PyUnicode_FromUnicode(NULL, ressize);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
723 if (res==NULL)
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
724 return NULL;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
725 for (p = startp+start, outp = PyUnicode_AS_UNICODE(res);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
726 p < startp+end; ++p) {
64590
fc76e8c617dc Issue #9804: ascii() now always represents unicode surrogate pairs as
Antoine Pitrou <solipsis@pitrou.net>
parents: 61466
diff changeset
727 Py_UCS4 c = (Py_UCS4) *p;
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
728 *outp++ = '\\';
64590
fc76e8c617dc Issue #9804: ascii() now always represents unicode surrogate pairs as
Antoine Pitrou <solipsis@pitrou.net>
parents: 61466
diff changeset
729 if (IS_SURROGATE_PAIR(p, startp+end)) {
fc76e8c617dc Issue #9804: ascii() now always represents unicode surrogate pairs as
Antoine Pitrou <solipsis@pitrou.net>
parents: 61466
diff changeset
730 c = ((*p & 0x3FF) << 10) + (*(p + 1) & 0x3FF) + 0x10000;
fc76e8c617dc Issue #9804: ascii() now always represents unicode surrogate pairs as
Antoine Pitrou <solipsis@pitrou.net>
parents: 61466
diff changeset
731 ++p;
fc76e8c617dc Issue #9804: ascii() now always represents unicode surrogate pairs as
Antoine Pitrou <solipsis@pitrou.net>
parents: 61466
diff changeset
732 }
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
733 if (c >= 0x00010000) {
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
734 *outp++ = 'U';
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
735 *outp++ = hexdigits[(c>>28)&0xf];
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
736 *outp++ = hexdigits[(c>>24)&0xf];
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
737 *outp++ = hexdigits[(c>>20)&0xf];
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
738 *outp++ = hexdigits[(c>>16)&0xf];
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
739 *outp++ = hexdigits[(c>>12)&0xf];
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
740 *outp++ = hexdigits[(c>>8)&0xf];
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
741 }
64590
fc76e8c617dc Issue #9804: ascii() now always represents unicode surrogate pairs as
Antoine Pitrou <solipsis@pitrou.net>
parents: 61466
diff changeset
742 else if (c >= 0x100) {
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
743 *outp++ = 'u';
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
744 *outp++ = hexdigits[(c>>12)&0xf];
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
745 *outp++ = hexdigits[(c>>8)&0xf];
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
746 }
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
747 else
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
748 *outp++ = 'x';
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
749 *outp++ = hexdigits[(c>>4)&0xf];
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
750 *outp++ = hexdigits[c&0xf];
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
751 }
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
752
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
753 restuple = Py_BuildValue("(On)", res, end);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
754 Py_DECREF(res);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
755 Py_DECREF(object);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
756 return restuple;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
757 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
758 else {
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
759 wrong_exception_type(exc);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
760 return NULL;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
761 }
64590
fc76e8c617dc Issue #9804: ascii() now always represents unicode surrogate pairs as
Antoine Pitrou <solipsis@pitrou.net>
parents: 61466
diff changeset
762 #undef IS_SURROGATE_PAIR
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
763 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
764
53687
52a89d723b6b Make PyCodec_SurrogateErrors static.
Martin v. Löwis <martin@v.loewis.de>
parents: 53681
diff changeset
765 /* This handler is declared static until someone demonstrates
52a89d723b6b Make PyCodec_SurrogateErrors static.
Martin v. Löwis <martin@v.loewis.de>
parents: 53681
diff changeset
766 a need to call it directly. */
52a89d723b6b Make PyCodec_SurrogateErrors static.
Martin v. Löwis <martin@v.loewis.de>
parents: 53681
diff changeset
767 static PyObject *
53965
fbcdae3e0ca1 Rename the surrogates error handler to surrogatepass.
Martin v. Löwis <martin@v.loewis.de>
parents: 53767
diff changeset
768 PyCodec_SurrogatePassErrors(PyObject *exc)
53681
52a77ef069cd Issue #3672: Reject surrogates in utf-8 codec; add surrogates error
Martin v. Löwis <martin@v.loewis.de>
parents: 49613
diff changeset
769 {
52a77ef069cd Issue #3672: Reject surrogates in utf-8 codec; add surrogates error
Martin v. Löwis <martin@v.loewis.de>
parents: 49613
diff changeset
770 PyObject *restuple;
52a77ef069cd Issue #3672: Reject surrogates in utf-8 codec; add surrogates error
Martin v. Löwis <martin@v.loewis.de>
parents: 49613
diff changeset
771 PyObject *object;
52a77ef069cd Issue #3672: Reject surrogates in utf-8 codec; add surrogates error
Martin v. Löwis <martin@v.loewis.de>
parents: 49613
diff changeset
772 Py_ssize_t start;
52a77ef069cd Issue #3672: Reject surrogates in utf-8 codec; add surrogates error
Martin v. Löwis <martin@v.loewis.de>
parents: 49613
diff changeset
773 Py_ssize_t end;
52a77ef069cd Issue #3672: Reject surrogates in utf-8 codec; add surrogates error
Martin v. Löwis <martin@v.loewis.de>
parents: 49613
diff changeset
774 PyObject *res;
52a77ef069cd Issue #3672: Reject surrogates in utf-8 codec; add surrogates error
Martin v. Löwis <martin@v.loewis.de>
parents: 49613
diff changeset
775 if (PyObject_IsInstance(exc, PyExc_UnicodeEncodeError)) {
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
776 Py_UNICODE *p;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
777 Py_UNICODE *startp;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
778 char *outp;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
779 if (PyUnicodeEncodeError_GetStart(exc, &start))
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
780 return NULL;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
781 if (PyUnicodeEncodeError_GetEnd(exc, &end))
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
782 return NULL;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
783 if (!(object = PyUnicodeEncodeError_GetObject(exc)))
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
784 return NULL;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
785 startp = PyUnicode_AS_UNICODE(object);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
786 res = PyBytes_FromStringAndSize(NULL, 3*(end-start));
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
787 if (!res) {
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
788 Py_DECREF(object);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
789 return NULL;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
790 }
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
791 outp = PyBytes_AsString(res);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
792 for (p = startp+start; p < startp+end; p++) {
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
793 Py_UNICODE ch = *p;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
794 if (ch < 0xd800 || ch > 0xdfff) {
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
795 /* Not a surrogate, fail with original exception */
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
796 PyErr_SetObject(PyExceptionInstance_Class(exc), exc);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
797 Py_DECREF(res);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
798 Py_DECREF(object);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
799 return NULL;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
800 }
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
801 *outp++ = (char)(0xe0 | (ch >> 12));
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
802 *outp++ = (char)(0x80 | ((ch >> 6) & 0x3f));
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
803 *outp++ = (char)(0x80 | (ch & 0x3f));
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
804 }
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
805 restuple = Py_BuildValue("(On)", res, end);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
806 Py_DECREF(res);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
807 Py_DECREF(object);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
808 return restuple;
53681
52a77ef069cd Issue #3672: Reject surrogates in utf-8 codec; add surrogates error
Martin v. Löwis <martin@v.loewis.de>
parents: 49613
diff changeset
809 }
52a77ef069cd Issue #3672: Reject surrogates in utf-8 codec; add surrogates error
Martin v. Löwis <martin@v.loewis.de>
parents: 49613
diff changeset
810 else if (PyObject_IsInstance(exc, PyExc_UnicodeDecodeError)) {
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
811 unsigned char *p;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
812 Py_UNICODE ch = 0;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
813 if (PyUnicodeDecodeError_GetStart(exc, &start))
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
814 return NULL;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
815 if (!(object = PyUnicodeDecodeError_GetObject(exc)))
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
816 return NULL;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
817 if (!(p = (unsigned char*)PyBytes_AsString(object))) {
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
818 Py_DECREF(object);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
819 return NULL;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
820 }
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
821 /* Try decoding a single surrogate character. If
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
822 there are more, let the codec call us again. */
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
823 p += start;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
824 if ((p[0] & 0xf0) == 0xe0 ||
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
825 (p[1] & 0xc0) == 0x80 ||
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
826 (p[2] & 0xc0) == 0x80) {
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
827 /* it's a three-byte code */
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
828 ch = ((p[0] & 0x0f) << 12) + ((p[1] & 0x3f) << 6) + (p[2] & 0x3f);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
829 if (ch < 0xd800 || ch > 0xdfff)
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
830 /* it's not a surrogate - fail */
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
831 ch = 0;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
832 }
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
833 Py_DECREF(object);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
834 if (ch == 0) {
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
835 PyErr_SetObject(PyExceptionInstance_Class(exc), exc);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
836 return NULL;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
837 }
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
838 return Py_BuildValue("(u#n)", &ch, 1, start+3);
53681
52a77ef069cd Issue #3672: Reject surrogates in utf-8 codec; add surrogates error
Martin v. Löwis <martin@v.loewis.de>
parents: 49613
diff changeset
839 }
52a77ef069cd Issue #3672: Reject surrogates in utf-8 codec; add surrogates error
Martin v. Löwis <martin@v.loewis.de>
parents: 49613
diff changeset
840 else {
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
841 wrong_exception_type(exc);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
842 return NULL;
53681
52a77ef069cd Issue #3672: Reject surrogates in utf-8 codec; add surrogates error
Martin v. Löwis <martin@v.loewis.de>
parents: 49613
diff changeset
843 }
52a77ef069cd Issue #3672: Reject surrogates in utf-8 codec; add surrogates error
Martin v. Löwis <martin@v.loewis.de>
parents: 49613
diff changeset
844 }
52a77ef069cd Issue #3672: Reject surrogates in utf-8 codec; add surrogates error
Martin v. Löwis <martin@v.loewis.de>
parents: 49613
diff changeset
845
53767
ac209f132b17 Issue #5915: Implement PEP 383, Non-decodable Bytes in
Martin v. Löwis <martin@v.loewis.de>
parents: 53687
diff changeset
846 static PyObject *
53966
7c29bfbfd352 Rename utf8b error handler to surrogateescape.
Martin v. Löwis <martin@v.loewis.de>
parents: 53965
diff changeset
847 PyCodec_SurrogateEscapeErrors(PyObject *exc)
53767
ac209f132b17 Issue #5915: Implement PEP 383, Non-decodable Bytes in
Martin v. Löwis <martin@v.loewis.de>
parents: 53687
diff changeset
848 {
ac209f132b17 Issue #5915: Implement PEP 383, Non-decodable Bytes in
Martin v. Löwis <martin@v.loewis.de>
parents: 53687
diff changeset
849 PyObject *restuple;
ac209f132b17 Issue #5915: Implement PEP 383, Non-decodable Bytes in
Martin v. Löwis <martin@v.loewis.de>
parents: 53687
diff changeset
850 PyObject *object;
ac209f132b17 Issue #5915: Implement PEP 383, Non-decodable Bytes in
Martin v. Löwis <martin@v.loewis.de>
parents: 53687
diff changeset
851 Py_ssize_t start;
ac209f132b17 Issue #5915: Implement PEP 383, Non-decodable Bytes in
Martin v. Löwis <martin@v.loewis.de>
parents: 53687
diff changeset
852 Py_ssize_t end;
ac209f132b17 Issue #5915: Implement PEP 383, Non-decodable Bytes in
Martin v. Löwis <martin@v.loewis.de>
parents: 53687
diff changeset
853 PyObject *res;
ac209f132b17 Issue #5915: Implement PEP 383, Non-decodable Bytes in
Martin v. Löwis <martin@v.loewis.de>
parents: 53687
diff changeset
854 if (PyObject_IsInstance(exc, PyExc_UnicodeEncodeError)) {
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
855 Py_UNICODE *p;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
856 Py_UNICODE *startp;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
857 char *outp;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
858 if (PyUnicodeEncodeError_GetStart(exc, &start))
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
859 return NULL;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
860 if (PyUnicodeEncodeError_GetEnd(exc, &end))
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
861 return NULL;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
862 if (!(object = PyUnicodeEncodeError_GetObject(exc)))
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
863 return NULL;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
864 startp = PyUnicode_AS_UNICODE(object);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
865 res = PyBytes_FromStringAndSize(NULL, end-start);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
866 if (!res) {
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
867 Py_DECREF(object);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
868 return NULL;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
869 }
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
870 outp = PyBytes_AsString(res);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
871 for (p = startp+start; p < startp+end; p++) {
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
872 Py_UNICODE ch = *p;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
873 if (ch < 0xdc80 || ch > 0xdcff) {
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
874 /* Not a UTF-8b surrogate, fail with original exception */
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
875 PyErr_SetObject(PyExceptionInstance_Class(exc), exc);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
876 Py_DECREF(res);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
877 Py_DECREF(object);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
878 return NULL;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
879 }
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
880 *outp++ = ch - 0xdc00;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
881 }
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
882 restuple = Py_BuildValue("(On)", res, end);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
883 Py_DECREF(res);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
884 Py_DECREF(object);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
885 return restuple;
53767
ac209f132b17 Issue #5915: Implement PEP 383, Non-decodable Bytes in
Martin v. Löwis <martin@v.loewis.de>
parents: 53687
diff changeset
886 }
ac209f132b17 Issue #5915: Implement PEP 383, Non-decodable Bytes in
Martin v. Löwis <martin@v.loewis.de>
parents: 53687
diff changeset
887 else if (PyObject_IsInstance(exc, PyExc_UnicodeDecodeError)) {
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
888 unsigned char *p;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
889 Py_UNICODE ch[4]; /* decode up to 4 bad bytes. */
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
890 int consumed = 0;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
891 if (PyUnicodeDecodeError_GetStart(exc, &start))
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
892 return NULL;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
893 if (PyUnicodeDecodeError_GetEnd(exc, &end))
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
894 return NULL;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
895 if (!(object = PyUnicodeDecodeError_GetObject(exc)))
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
896 return NULL;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
897 if (!(p = (unsigned char*)PyBytes_AsString(object))) {
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
898 Py_DECREF(object);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
899 return NULL;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
900 }
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
901 while (consumed < 4 && consumed < end-start) {
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
902 /* Refuse to escape ASCII bytes. */
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
903 if (p[start+consumed] < 128)
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
904 break;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
905 ch[consumed] = 0xdc00 + p[start+consumed];
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
906 consumed++;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
907 }
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
908 Py_DECREF(object);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
909 if (!consumed) {
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
910 /* codec complained about ASCII byte. */
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
911 PyErr_SetObject(PyExceptionInstance_Class(exc), exc);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
912 return NULL;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
913 }
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
914 return Py_BuildValue("(u#n)", ch, consumed, start+consumed);
53767
ac209f132b17 Issue #5915: Implement PEP 383, Non-decodable Bytes in
Martin v. Löwis <martin@v.loewis.de>
parents: 53687
diff changeset
915 }
ac209f132b17 Issue #5915: Implement PEP 383, Non-decodable Bytes in
Martin v. Löwis <martin@v.loewis.de>
parents: 53687
diff changeset
916 else {
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
917 wrong_exception_type(exc);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
918 return NULL;
53767
ac209f132b17 Issue #5915: Implement PEP 383, Non-decodable Bytes in
Martin v. Löwis <martin@v.loewis.de>
parents: 53687
diff changeset
919 }
ac209f132b17 Issue #5915: Implement PEP 383, Non-decodable Bytes in
Martin v. Löwis <martin@v.loewis.de>
parents: 53687
diff changeset
920 }
ac209f132b17 Issue #5915: Implement PEP 383, Non-decodable Bytes in
Martin v. Löwis <martin@v.loewis.de>
parents: 53687
diff changeset
921
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
922
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
923 static PyObject *strict_errors(PyObject *self, PyObject *exc)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
924 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
925 return PyCodec_StrictErrors(exc);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
926 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
927
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
928
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
929 static PyObject *ignore_errors(PyObject *self, PyObject *exc)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
930 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
931 return PyCodec_IgnoreErrors(exc);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
932 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
933
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
934
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
935 static PyObject *replace_errors(PyObject *self, PyObject *exc)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
936 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
937 return PyCodec_ReplaceErrors(exc);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
938 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
939
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
940
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
941 static PyObject *xmlcharrefreplace_errors(PyObject *self, PyObject *exc)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
942 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
943 return PyCodec_XMLCharRefReplaceErrors(exc);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
944 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
945
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
946
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
947 static PyObject *backslashreplace_errors(PyObject *self, PyObject *exc)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
948 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
949 return PyCodec_BackslashReplaceErrors(exc);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
950 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
951
53965
fbcdae3e0ca1 Rename the surrogates error handler to surrogatepass.
Martin v. Löwis <martin@v.loewis.de>
parents: 53767
diff changeset
952 static PyObject *surrogatepass_errors(PyObject *self, PyObject *exc)
53681
52a77ef069cd Issue #3672: Reject surrogates in utf-8 codec; add surrogates error
Martin v. Löwis <martin@v.loewis.de>
parents: 49613
diff changeset
953 {
53965
fbcdae3e0ca1 Rename the surrogates error handler to surrogatepass.
Martin v. Löwis <martin@v.loewis.de>
parents: 53767
diff changeset
954 return PyCodec_SurrogatePassErrors(exc);
53681
52a77ef069cd Issue #3672: Reject surrogates in utf-8 codec; add surrogates error
Martin v. Löwis <martin@v.loewis.de>
parents: 49613
diff changeset
955 }
52a77ef069cd Issue #3672: Reject surrogates in utf-8 codec; add surrogates error
Martin v. Löwis <martin@v.loewis.de>
parents: 49613
diff changeset
956
53966
7c29bfbfd352 Rename utf8b error handler to surrogateescape.
Martin v. Löwis <martin@v.loewis.de>
parents: 53965
diff changeset
957 static PyObject *surrogateescape_errors(PyObject *self, PyObject *exc)
53767
ac209f132b17 Issue #5915: Implement PEP 383, Non-decodable Bytes in
Martin v. Löwis <martin@v.loewis.de>
parents: 53687
diff changeset
958 {
53966
7c29bfbfd352 Rename utf8b error handler to surrogateescape.
Martin v. Löwis <martin@v.loewis.de>
parents: 53965
diff changeset
959 return PyCodec_SurrogateEscapeErrors(exc);
53767
ac209f132b17 Issue #5915: Implement PEP 383, Non-decodable Bytes in
Martin v. Löwis <martin@v.loewis.de>
parents: 53687
diff changeset
960 }
ac209f132b17 Issue #5915: Implement PEP 383, Non-decodable Bytes in
Martin v. Löwis <martin@v.loewis.de>
parents: 53687
diff changeset
961
28325
850626d47574 Fixed SF bug #663074. The codec system was using global static
Gustavo Niemeyer <gustavo@niemeyer.net>
parents: 27898
diff changeset
962 static int _PyCodecRegistry_Init(void)
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
963 {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
964 static struct {
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
965 char *name;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
966 PyMethodDef def;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
967 } methods[] =
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
968 {
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
969 {
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
970 "strict",
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
971 {
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
972 "strict_errors",
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
973 strict_errors,
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
974 METH_O,
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
975 PyDoc_STR("Implements the 'strict' error handling, which "
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
976 "raises a UnicodeError on coding errors.")
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
977 }
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
978 },
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
979 {
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
980 "ignore",
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
981 {
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
982 "ignore_errors",
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
983 ignore_errors,
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
984 METH_O,
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
985 PyDoc_STR("Implements the 'ignore' error handling, which "
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
986 "ignores malformed data and continues.")
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
987 }
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
988 },
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
989 {
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
990 "replace",
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
991 {
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
992 "replace_errors",
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
993 replace_errors,
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
994 METH_O,
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
995 PyDoc_STR("Implements the 'replace' error handling, which "
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
996 "replaces malformed data with a replacement marker.")
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
997 }
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
998 },
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
999 {
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
1000 "xmlcharrefreplace",
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
1001 {
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
1002 "xmlcharrefreplace_errors",
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
1003 xmlcharrefreplace_errors,
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
1004 METH_O,
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
1005 PyDoc_STR("Implements the 'xmlcharrefreplace' error handling, "
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
1006 "which replaces an unencodable character with the "
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
1007 "appropriate XML character reference.")
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
1008 }
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
1009 },
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
1010 {
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
1011 "backslashreplace",
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
1012 {
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
1013 "backslashreplace_errors",
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
1014 backslashreplace_errors,
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
1015 METH_O,
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
1016 PyDoc_STR("Implements the 'backslashreplace' error handling, "
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
1017 "which replaces an unencodable character with a "
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
1018 "backslashed escape sequence.")
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
1019 }
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
1020 },
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
1021 {
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
1022 "surrogatepass",
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
1023 {
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
1024 "surrogatepass",
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
1025 surrogatepass_errors,
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
1026 METH_O
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
1027 }
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
1028 },
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
1029 {
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
1030 "surrogateescape",
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
1031 {
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
1032 "surrogateescape",
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
1033 surrogateescape_errors,
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
1034 METH_O
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
1035 }
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
1036 }
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
1037 };
28325
850626d47574 Fixed SF bug #663074. The codec system was using global static
Gustavo Niemeyer <gustavo@niemeyer.net>
parents: 27898
diff changeset
1038
31797
846b3ee7e106 Changed random calls to PyThreadState_Get() to use the macro
Nicholas Bastin <nick.bastin@gmail.com>
parents: 31273
diff changeset
1039 PyInterpreterState *interp = PyThreadState_GET()->interp;
28325
850626d47574 Fixed SF bug #663074. The codec system was using global static
Gustavo Niemeyer <gustavo@niemeyer.net>
parents: 27898
diff changeset
1040 PyObject *mod;
32533
201986d6d9d6 Fix a couple of signed/unsigned comparison warnings
Neal Norwitz <nnorwitz@gmail.com>
parents: 31797
diff changeset
1041 unsigned i;
28325
850626d47574 Fixed SF bug #663074. The codec system was using global static
Gustavo Niemeyer <gustavo@niemeyer.net>
parents: 27898
diff changeset
1042
850626d47574 Fixed SF bug #663074. The codec system was using global static
Gustavo Niemeyer <gustavo@niemeyer.net>
parents: 27898
diff changeset
1043 if (interp->codec_search_path != NULL)
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
1044 return 0;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
1045
28325
850626d47574 Fixed SF bug #663074. The codec system was using global static
Gustavo Niemeyer <gustavo@niemeyer.net>
parents: 27898
diff changeset
1046 interp->codec_search_path = PyList_New(0);
850626d47574 Fixed SF bug #663074. The codec system was using global static
Gustavo Niemeyer <gustavo@niemeyer.net>
parents: 27898
diff changeset
1047 interp->codec_search_cache = PyDict_New();
850626d47574 Fixed SF bug #663074. The codec system was using global static
Gustavo Niemeyer <gustavo@niemeyer.net>
parents: 27898
diff changeset
1048 interp->codec_error_registry = PyDict_New();
850626d47574 Fixed SF bug #663074. The codec system was using global static
Gustavo Niemeyer <gustavo@niemeyer.net>
parents: 27898
diff changeset
1049
850626d47574 Fixed SF bug #663074. The codec system was using global static
Gustavo Niemeyer <gustavo@niemeyer.net>
parents: 27898
diff changeset
1050 if (interp->codec_error_registry) {
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
1051 for (i = 0; i < sizeof(methods)/sizeof(methods[0]); ++i) {
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
1052 PyObject *func = PyCFunction_New(&methods[i].def, NULL);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
1053 int res;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
1054 if (!func)
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
1055 Py_FatalError("can't initialize codec error registry");
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
1056 res = PyCodec_RegisterError(methods[i].name, func);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
1057 Py_DECREF(func);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
1058 if (res)
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
1059 Py_FatalError("can't initialize codec error registry");
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
1060 }
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 24307
diff changeset
1061 }
28325
850626d47574 Fixed SF bug #663074. The codec system was using global static
Gustavo Niemeyer <gustavo@niemeyer.net>
parents: 27898
diff changeset
1062
850626d47574 Fixed SF bug #663074. The codec system was using global static
Gustavo Niemeyer <gustavo@niemeyer.net>
parents: 27898
diff changeset
1063 if (interp->codec_search_path == NULL ||
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
1064 interp->codec_search_cache == NULL ||
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
1065 interp->codec_error_registry == NULL)
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
1066 Py_FatalError("can't initialize codec registry");
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
1067
44442
fa2f7183b1af More PyImport_ImportModule -> PyImport_ImportModuleNoBlock
Christian Heimes <christian@cheimes.de>
parents: 44351
diff changeset
1068 mod = PyImport_ImportModuleNoBlock("encodings");
28325
850626d47574 Fixed SF bug #663074. The codec system was using global static
Gustavo Niemeyer <gustavo@niemeyer.net>
parents: 27898
diff changeset
1069 if (mod == NULL) {
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 56894
diff changeset
1070 return -1;
28325
850626d47574 Fixed SF bug #663074. The codec system was using global static
Gustavo Niemeyer <gustavo@niemeyer.net>
parents: 27898
diff changeset
1071 }
850626d47574 Fixed SF bug #663074. The codec system was using global static
Gustavo Niemeyer <gustavo@niemeyer.net>
parents: 27898
diff changeset
1072 Py_DECREF(mod);
49613
efef5fc45a9a Issue 3723: Fixed initialization of subinterpreters
Christian Heimes <christian@cheimes.de>
parents: 47496
diff changeset
1073 interp->codecs_initialized = 1;
28325
850626d47574 Fixed SF bug #663074. The codec system was using global static
Gustavo Niemeyer <gustavo@niemeyer.net>
parents: 27898
diff changeset
1074 return 0;
11972
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre
Guido van Rossum <guido@python.org>
parents:
diff changeset
1075 }