cpython-lru_cache-weakref: Python/codecs.c annotate

annotate Python/codecs.c @ 12584:cde4da18c4fa legacy-trunk

Fast NonRecursiveMutex support by Yakov Markovitch, markovitch@iso.ru, who wrote: Here's the new version of thread_nt.h. More particular, there is a new version of thread lock that uses kernel object (e.g. semaphore) only in case of contention; in other case it simply uses interlocked functions, which are faster by the order of magnitude. It doesn't make much difference without threads present, but as soon as thread machinery initialised and (mostly) the interpreter global lock is on, difference becomes tremendous. I've included a small script, which initialises threads and launches pystone. With original thread_nt.h, Pystone results with initialised threads are twofold worse then w/o threads. With the new version, only 10% worse. I have used this patch for about 6 months (with threaded and non-threaded applications). It works remarkably well (though I'd desperately prefer Python was free-threaded; I hope, it will soon).

author	Guido van Rossum <guido@python.org>
date	Thu, 04 May 2000 18:47:15 +0000
parents	70154e2e9c89
children	2a53f5dc0590

rev	line source
11972 d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	1 /* ------------------------------------------------------------------------
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	2
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	3 Python Codec Registry and support functions
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	4
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	5 Written by Marc-Andre Lemburg (mal@lemburg.com).
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	6
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	7 (c) Copyright CNRI, All Rights Reserved. NO WARRANTY.
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	8
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	9 ------------------------------------------------------------------------ */
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	10
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	11 #include "Python.h"
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	12 #include <ctype.h>
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	13
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	14 /* --- Globals ------------------------------------------------------------ */
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	15
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	16 static PyObject *_PyCodec_SearchPath;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	17 static PyObject *_PyCodec_SearchCache;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	18
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	19 /* Flag used for lazy import of the standard encodings package */
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	20 static int import_encodings_called = 0;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	21
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	22 /* --- Codec Registry ----------------------------------------------------- */
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	23
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	24 /* Import the standard encodings package which will register the first
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	25 codec search function.
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	26
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	27 This is done in a lazy way so that the Unicode implementation does
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	28 not downgrade startup time of scripts not needing it.
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	29
12195 bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup Guido van Rossum <guido@python.org> parents: 12048 diff changeset	30 ImportErrors are silently ignored by this function. Only one try is
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup Guido van Rossum <guido@python.org> parents: 12048 diff changeset	31 made.
11972 d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	32
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	33 */
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	34
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	35 static
12195 bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup Guido van Rossum <guido@python.org> parents: 12048 diff changeset	36 int import_encodings()
11972 d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	37 {
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	38 PyObject *mod;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	39
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	40 import_encodings_called = 1;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	41 mod = PyImport_ImportModule("encodings");
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	42 if (mod == NULL) {
12195 bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup Guido van Rossum <guido@python.org> parents: 12048 diff changeset	43 if (PyErr_ExceptionMatches(PyExc_ImportError)) {
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup Guido van Rossum <guido@python.org> parents: 12048 diff changeset	44 /* Ignore ImportErrors... this is done so that
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup Guido van Rossum <guido@python.org> parents: 12048 diff changeset	45 distributions can disable the encodings package. Note
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup Guido van Rossum <guido@python.org> parents: 12048 diff changeset	46 that other errors are not masked, e.g. SystemErrors
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup Guido van Rossum <guido@python.org> parents: 12048 diff changeset	47 raised to inform the user of an error in the Python
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup Guido van Rossum <guido@python.org> parents: 12048 diff changeset	48 configuration are still reported back to the user. */
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup Guido van Rossum <guido@python.org> parents: 12048 diff changeset	49 PyErr_Clear();
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup Guido van Rossum <guido@python.org> parents: 12048 diff changeset	50 return 0;
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup Guido van Rossum <guido@python.org> parents: 12048 diff changeset	51 }
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup Guido van Rossum <guido@python.org> parents: 12048 diff changeset	52 return -1;
11972 d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	53 }
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	54 Py_DECREF(mod);
12195 bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup Guido van Rossum <guido@python.org> parents: 12048 diff changeset	55 return 0;
11972 d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	56 }
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	57
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	58 /* Register a new codec search function.
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	59
12195 bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup Guido van Rossum <guido@python.org> parents: 12048 diff changeset	60 As side effect, this tries to load the encodings package, if not
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup Guido van Rossum <guido@python.org> parents: 12048 diff changeset	61 yet done, to make sure that it is always first in the list of
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup Guido van Rossum <guido@python.org> parents: 12048 diff changeset	62 search functions.
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup Guido van Rossum <guido@python.org> parents: 12048 diff changeset	63
11972 d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	64 The search_function's refcount is incremented by this function. */
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	65
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	66 int PyCodec_Register(PyObject *search_function)
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	67 {
12195 bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup Guido van Rossum <guido@python.org> parents: 12048 diff changeset	68 if (!import_encodings_called) {
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup Guido van Rossum <guido@python.org> parents: 12048 diff changeset	69 if (import_encodings())
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup Guido van Rossum <guido@python.org> parents: 12048 diff changeset	70 goto onError;
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup Guido van Rossum <guido@python.org> parents: 12048 diff changeset	71 }
11972 d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	72 if (search_function == NULL) {
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	73 PyErr_BadArgument();
12195 bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup Guido van Rossum <guido@python.org> parents: 12048 diff changeset	74 goto onError;
11972 d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	75 }
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	76 if (!PyCallable_Check(search_function)) {
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	77 PyErr_SetString(PyExc_TypeError,
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	78 "argument must be callable");
12195 bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup Guido van Rossum <guido@python.org> parents: 12048 diff changeset	79 goto onError;
11972 d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	80 }
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	81 return PyList_Append(_PyCodec_SearchPath, search_function);
12195 bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup Guido van Rossum <guido@python.org> parents: 12048 diff changeset	82
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup Guido van Rossum <guido@python.org> parents: 12048 diff changeset	83 onError:
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup Guido van Rossum <guido@python.org> parents: 12048 diff changeset	84 return -1;
11972 d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	85 }
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	86
12248 70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12195 diff changeset	87 /* Convert a string to a normalized Python string: all characters are
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12195 diff changeset	88 converted to lower case, spaces are replaced with underscores. */
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12195 diff changeset	89
11972 d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	90 static
12248 70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12195 diff changeset	91 PyObject normalizestring(const char string)
11972 d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	92 {
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	93 register int i;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	94 int len = strlen(string);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	95 char *p;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	96 PyObject *v;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	97
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	98 v = PyString_FromStringAndSize(NULL, len);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	99 if (v == NULL)
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	100 return NULL;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	101 p = PyString_AS_STRING(v);
12248 70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12195 diff changeset	102 for (i = 0; i < len; i++) {
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12195 diff changeset	103 register char ch = string[i];
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12195 diff changeset	104 if (ch == ' ')
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12195 diff changeset	105 ch = '-';
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12195 diff changeset	106 else
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12195 diff changeset	107 ch = tolower(ch);
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12195 diff changeset	108 p[i] = ch;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12195 diff changeset	109 }
11972 d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	110 return v;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	111 }
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	112
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	113 /* Lookup the given encoding and return a tuple providing the codec
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	114 facilities.
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	115
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	116 The encoding string is looked up converted to all lower-case
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	117 characters. This makes encodings looked up through this mechanism
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	118 effectively case-insensitive.
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	119
12195 bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup Guido van Rossum <guido@python.org> parents: 12048 diff changeset	120 If no codec is found, a KeyError is set and NULL returned.
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup Guido van Rossum <guido@python.org> parents: 12048 diff changeset	121
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup Guido van Rossum <guido@python.org> parents: 12048 diff changeset	122 As side effect, this tries to load the encodings package, if not
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup Guido van Rossum <guido@python.org> parents: 12048 diff changeset	123 yet done. This is part of the lazy load strategy for the encodings
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup Guido van Rossum <guido@python.org> parents: 12048 diff changeset	124 package.
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup Guido van Rossum <guido@python.org> parents: 12048 diff changeset	125
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup Guido van Rossum <guido@python.org> parents: 12048 diff changeset	126 */
11972 d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	127
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	128 PyObject _PyCodec_Lookup(const char encoding)
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	129 {
12048 f87b56eebacc Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12025 diff changeset	130 PyObject result, args = NULL, *v;
11972 d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	131 int i, len;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	132
12195 bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup Guido van Rossum <guido@python.org> parents: 12048 diff changeset	133 if (_PyCodec_SearchCache == NULL \|\|
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup Guido van Rossum <guido@python.org> parents: 12048 diff changeset	134 _PyCodec_SearchPath == NULL) {
12025 7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said: Barry Warsaw <barry@python.org> parents: 11972 diff changeset	135 PyErr_SetString(PyExc_SystemError,
7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said: Barry Warsaw <barry@python.org> parents: 11972 diff changeset	136 "codec module not properly initialized");
7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said: Barry Warsaw <barry@python.org> parents: 11972 diff changeset	137 goto onError;
7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said: Barry Warsaw <barry@python.org> parents: 11972 diff changeset	138 }
12195 bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup Guido van Rossum <guido@python.org> parents: 12048 diff changeset	139 if (!import_encodings_called) {
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup Guido van Rossum <guido@python.org> parents: 12048 diff changeset	140 if (import_encodings())
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup Guido van Rossum <guido@python.org> parents: 12048 diff changeset	141 goto onError;
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup Guido van Rossum <guido@python.org> parents: 12048 diff changeset	142 }
11972 d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	143
12248 70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12195 diff changeset	144 /* Convert the encoding to a normalized Python string: all
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12195 diff changeset	145 characters are converted to lower case, spaces and hypens are
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12195 diff changeset	146 replaced with underscores. */
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12195 diff changeset	147 v = normalizestring(encoding);
11972 d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	148 if (v == NULL)
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	149 goto onError;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	150 PyString_InternInPlace(&v);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	151
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	152 /* First, try to lookup the name in the registry dictionary */
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	153 result = PyDict_GetItem(_PyCodec_SearchCache, v);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	154 if (result != NULL) {
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	155 Py_INCREF(result);
12025 7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said: Barry Warsaw <barry@python.org> parents: 11972 diff changeset	156 Py_DECREF(v);
11972 d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	157 return result;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	158 }
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	159
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	160 /* Next, scan the search functions in order of registration */
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	161 args = PyTuple_New(1);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	162 if (args == NULL)
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	163 goto onError;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	164 PyTuple_SET_ITEM(args,0,v);
12048 f87b56eebacc Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12025 diff changeset	165
f87b56eebacc Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12025 diff changeset	166 len = PyList_Size(_PyCodec_SearchPath);
f87b56eebacc Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12025 diff changeset	167 if (len < 0)
f87b56eebacc Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12025 diff changeset	168 goto onError;
12195 bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup Guido van Rossum <guido@python.org> parents: 12048 diff changeset	169 if (len == 0) {
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup Guido van Rossum <guido@python.org> parents: 12048 diff changeset	170 PyErr_SetString(PyExc_LookupError,
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup Guido van Rossum <guido@python.org> parents: 12048 diff changeset	171 "no codec search functions registered: "
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup Guido van Rossum <guido@python.org> parents: 12048 diff changeset	172 "can't find encoding");
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup Guido van Rossum <guido@python.org> parents: 12048 diff changeset	173 goto onError;
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup Guido van Rossum <guido@python.org> parents: 12048 diff changeset	174 }
11972 d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	175
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	176 for (i = 0; i < len; i++) {
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	177 PyObject *func;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	178
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	179 func = PyList_GetItem(_PyCodec_SearchPath, i);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	180 if (func == NULL)
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	181 goto onError;
12048 f87b56eebacc Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12025 diff changeset	182 result = PyEval_CallObject(func, args);
11972 d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	183 if (result == NULL)
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	184 goto onError;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	185 if (result == Py_None) {
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	186 Py_DECREF(result);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	187 continue;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	188 }
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	189 if (!PyTuple_Check(result) \|\| PyTuple_GET_SIZE(result) != 4) {
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	190 PyErr_SetString(PyExc_TypeError,
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	191 "codec search functions must return 4-tuples");
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	192 Py_DECREF(result);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	193 goto onError;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	194 }
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	195 break;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	196 }
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	197 if (i == len) {
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	198 /* XXX Perhaps we should cache misses too ? */
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	199 PyErr_SetString(PyExc_LookupError,
12025 7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said: Barry Warsaw <barry@python.org> parents: 11972 diff changeset	200 "unknown encoding");
11972 d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	201 goto onError;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	202 }
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	203
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	204 /* Cache and return the result */
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	205 PyDict_SetItem(_PyCodec_SearchCache, v, result);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	206 Py_DECREF(args);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	207 return result;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	208
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	209 onError:
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	210 Py_XDECREF(args);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	211 return NULL;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	212 }
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	213
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	214 static
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	215 PyObject args_tuple(PyObject object,
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	216 const char *errors)
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	217 {
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	218 PyObject *args;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	219
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	220 args = PyTuple_New(1 + (errors != NULL));
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	221 if (args == NULL)
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	222 return NULL;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	223 Py_INCREF(object);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	224 PyTuple_SET_ITEM(args,0,object);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	225 if (errors) {
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	226 PyObject *v;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	227
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	228 v = PyString_FromString(errors);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	229 if (v == NULL) {
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	230 Py_DECREF(args);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	231 return NULL;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	232 }
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	233 PyTuple_SET_ITEM(args, 1, v);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	234 }
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	235 return args;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	236 }
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	237
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	238 /* Build a codec by calling factory(stream[,errors]) or just
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	239 factory(errors) depending on whether the given parameters are
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	240 non-NULL. */
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	241
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	242 static
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	243 PyObject build_stream_codec(PyObject factory,
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	244 PyObject *stream,
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	245 const char *errors)
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	246 {
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	247 PyObject args, codec;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	248
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	249 args = args_tuple(stream, errors);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	250 if (args == NULL)
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	251 return NULL;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	252
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	253 codec = PyEval_CallObject(factory, args);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	254 Py_DECREF(args);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	255 return codec;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	256 }
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	257
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	258 /* Convenience APIs to query the Codec registry.
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	259
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	260 All APIs return a codec object with incremented refcount.
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	261
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	262 */
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	263
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	264 PyObject PyCodec_Encoder(const char encoding)
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	265 {
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	266 PyObject *codecs;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	267 PyObject *v;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	268
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	269 codecs = _PyCodec_Lookup(encoding);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	270 if (codecs == NULL)
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	271 goto onError;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	272 v = PyTuple_GET_ITEM(codecs,0);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	273 Py_INCREF(v);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	274 return v;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	275
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	276 onError:
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	277 return NULL;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	278 }
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	279
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	280 PyObject PyCodec_Decoder(const char encoding)
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	281 {
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	282 PyObject *codecs;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	283 PyObject *v;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	284
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	285 codecs = _PyCodec_Lookup(encoding);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	286 if (codecs == NULL)
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	287 goto onError;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	288 v = PyTuple_GET_ITEM(codecs,1);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	289 Py_INCREF(v);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	290 return v;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	291
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	292 onError:
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	293 return NULL;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	294 }
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	295
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	296 PyObject PyCodec_StreamReader(const char encoding,
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	297 PyObject *stream,
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	298 const char *errors)
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	299 {
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	300 PyObject *codecs;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	301
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	302 codecs = _PyCodec_Lookup(encoding);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	303 if (codecs == NULL)
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	304 goto onError;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	305 return build_stream_codec(PyTuple_GET_ITEM(codecs,2),stream,errors);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	306
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	307 onError:
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	308 return NULL;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	309 }
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	310
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	311 PyObject PyCodec_StreamWriter(const char encoding,
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	312 PyObject *stream,
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	313 const char *errors)
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	314 {
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	315 PyObject *codecs;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	316
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	317 codecs = _PyCodec_Lookup(encoding);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	318 if (codecs == NULL)
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	319 goto onError;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	320 return build_stream_codec(PyTuple_GET_ITEM(codecs,3),stream,errors);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	321
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	322 onError:
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	323 return NULL;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	324 }
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	325
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	326 /* Encode an object (e.g. an Unicode object) using the given encoding
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	327 and return the resulting encoded object (usually a Python string).
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	328
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	329 errors is passed to the encoder factory as argument if non-NULL. */
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	330
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	331 PyObject PyCodec_Encode(PyObject object,
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	332 const char *encoding,
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	333 const char *errors)
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	334 {
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	335 PyObject *encoder = NULL;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	336 PyObject args = NULL, result;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	337 PyObject *v;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	338
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	339 encoder = PyCodec_Encoder(encoding);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	340 if (encoder == NULL)
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	341 goto onError;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	342
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	343 args = args_tuple(object, errors);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	344 if (args == NULL)
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	345 goto onError;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	346
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	347 result = PyEval_CallObject(encoder,args);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	348 if (result == NULL)
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	349 goto onError;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	350
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	351 if (!PyTuple_Check(result) \|\|
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	352 PyTuple_GET_SIZE(result) != 2) {
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	353 PyErr_SetString(PyExc_TypeError,
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	354 "encoder must return a tuple (object,integer)");
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	355 goto onError;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	356 }
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	357 v = PyTuple_GET_ITEM(result,0);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	358 Py_INCREF(v);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	359 /* We don't check or use the second (integer) entry. */
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	360
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	361 Py_DECREF(args);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	362 Py_DECREF(encoder);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	363 Py_DECREF(result);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	364 return v;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	365
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	366 onError:
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	367 Py_XDECREF(args);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	368 Py_XDECREF(encoder);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	369 return NULL;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	370 }
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	371
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	372 /* Decode an object (usually a Python string) using the given encoding
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	373 and return an equivalent object (e.g. an Unicode object).
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	374
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	375 errors is passed to the decoder factory as argument if non-NULL. */
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	376
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	377 PyObject PyCodec_Decode(PyObject object,
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	378 const char *encoding,
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	379 const char *errors)
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	380 {
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	381 PyObject *decoder = NULL;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	382 PyObject args = NULL, result = NULL;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	383 PyObject *v;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	384
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	385 decoder = PyCodec_Decoder(encoding);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	386 if (decoder == NULL)
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	387 goto onError;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	388
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	389 args = args_tuple(object, errors);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	390 if (args == NULL)
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	391 goto onError;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	392
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	393 result = PyEval_CallObject(decoder,args);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	394 if (result == NULL)
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	395 goto onError;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	396 if (!PyTuple_Check(result) \|\|
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	397 PyTuple_GET_SIZE(result) != 2) {
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	398 PyErr_SetString(PyExc_TypeError,
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	399 "decoder must return a tuple (object,integer)");
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	400 goto onError;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	401 }
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	402 v = PyTuple_GET_ITEM(result,0);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	403 Py_INCREF(v);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	404 /* We don't check or use the second (integer) entry. */
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	405
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	406 Py_DECREF(args);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	407 Py_DECREF(decoder);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	408 Py_DECREF(result);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	409 return v;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	410
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	411 onError:
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	412 Py_XDECREF(args);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	413 Py_XDECREF(decoder);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	414 Py_XDECREF(result);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	415 return NULL;
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	416 }
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	417
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	418 void _PyCodecRegistry_Init()
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	419 {
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	420 if (_PyCodec_SearchPath == NULL)
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	421 _PyCodec_SearchPath = PyList_New(0);
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	422 if (_PyCodec_SearchCache == NULL)
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	423 _PyCodec_SearchCache = PyDict_New();
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	424 if (_PyCodec_SearchPath == NULL \|\|
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	425 _PyCodec_SearchCache == NULL)
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	426 Py_FatalError("can't intialize codec registry");
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	427 }
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	428
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	429 void _PyCodecRegistry_Fini()
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	430 {
d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	431 Py_XDECREF(_PyCodec_SearchPath);
12025 7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said: Barry Warsaw <barry@python.org> parents: 11972 diff changeset	432 _PyCodec_SearchPath = NULL;
11972 d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	433 Py_XDECREF(_PyCodec_SearchCache);
12025 7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said: Barry Warsaw <barry@python.org> parents: 11972 diff changeset	434 _PyCodec_SearchCache = NULL;
11972 d0e06efb3165 Python Codec Registry and support functions, written by Marc-Andre Guido van Rossum <guido@python.org> parents: diff changeset	435 }

Mercurial > cpython-lru_cache-weakref

annotate Python/codecs.c @ 12584:cde4da18c4fa legacy-trunk