Mercurial > cpython-lru_cache-weakref

annotate Doc/lib/libcodecs.tex @ 27998:e27f58a50737 legacy-trunk

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
SF bug #690377: Dud footnote reference in distutils docs
author Raymond Hettinger <python@rcn.com>
date Fri, 21 Feb 2003 02:47:09 +0000
parents c267f28724dd
children d41d8de9768f
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
12260
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
1 \section{\module{codecs} ---
12274
559b25a79032 Make sure the \declaremodule uses the right name for the module!
Fred Drake <fdrake@acm.org>
parents: 12260
diff changeset
2 Codec registry and base classes}
12260
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
3
12274
559b25a79032 Make sure the \declaremodule uses the right name for the module!
Fred Drake <fdrake@acm.org>
parents: 12260
diff changeset
4 \declaremodule{standard}{codecs}
12260
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
5 \modulesynopsis{Encode and decode data and streams.}
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
6 \moduleauthor{Marc-Andre Lemburg}{mal@lemburg.com}
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
7 \sectionauthor{Marc-Andre Lemburg}{mal@lemburg.com}
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
8
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
9
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
10 \index{Unicode}
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
11 \index{Codecs}
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
12 \indexii{Codecs}{encode}
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
13 \indexii{Codecs}{decode}
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
14 \index{streams}
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
15 \indexii{stackable}{streams}
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
16
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
17
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
18 This module defines base classes for standard Python codecs (encoders
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
19 and decoders) and provides access to the internal Python codec
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 23658
diff changeset
20 registry which manages the codec and error handling lookup process.
12260
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
21
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
22 It defines the following functions:
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
23
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
24 \begin{funcdesc}{register}{search_function}
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
25 Register a codec search function. Search functions are expected to
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
26 take one argument, the encoding name in all lower case letters, and
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
27 return a tuple of functions \code{(\var{encoder}, \var{decoder}, \var{stream_reader},
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
28 \var{stream_writer})} taking the following arguments:
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
29
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
30 \var{encoder} and \var{decoder}: These must be functions or methods
15186
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
31 which have the same interface as the
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
32 \method{encode()}/\method{decode()} methods of Codec instances (see
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
33 Codec Interface). The functions/methods are expected to work in a
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
34 stateless mode.
12260
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
35
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
36 \var{stream_reader} and \var{stream_writer}: These have to be
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
37 factory functions providing the following interface:
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
38
15186
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
39 \code{factory(\var{stream}, \var{errors}='strict')}
12260
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
40
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
41 The factory functions must return objects providing the interfaces
12274
559b25a79032 Make sure the \declaremodule uses the right name for the module!
Fred Drake <fdrake@acm.org>
parents: 12260
diff changeset
42 defined by the base classes \class{StreamWriter} and
559b25a79032 Make sure the \declaremodule uses the right name for the module!
Fred Drake <fdrake@acm.org>
parents: 12260
diff changeset
43 \class{StreamReader}, respectively. Stream codecs can maintain
559b25a79032 Make sure the \declaremodule uses the right name for the module!
Fred Drake <fdrake@acm.org>
parents: 12260
diff changeset
44 state.
12260
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
45
12274
559b25a79032 Make sure the \declaremodule uses the right name for the module!
Fred Drake <fdrake@acm.org>
parents: 12260
diff changeset
46 Possible values for errors are \code{'strict'} (raise an exception
559b25a79032 Make sure the \declaremodule uses the right name for the module!
Fred Drake <fdrake@acm.org>
parents: 12260
diff changeset
47 in case of an encoding error), \code{'replace'} (replace malformed
26294
6d0cd80e2804 Document additional error handling names available through PEP 293.
Walter Dörwald <walter@livinglogic.de>
parents: 26099
diff changeset
48 data with a suitable replacement marker, such as \character{?}),
12274
559b25a79032 Make sure the \declaremodule uses the right name for the module!
Fred Drake <fdrake@acm.org>
parents: 12260
diff changeset
49 \code{'ignore'} (ignore malformed data and continue without further
26294
6d0cd80e2804 Document additional error handling names available through PEP 293.
Walter Dörwald <walter@livinglogic.de>
parents: 26099
diff changeset
50 notice), \code{'xmlcharrefreplace'} (replace with the appropriate XML
6d0cd80e2804 Document additional error handling names available through PEP 293.
Walter Dörwald <walter@livinglogic.de>
parents: 26099
diff changeset
51 character reference (for encoding only)) and \code{'backslashreplace'}
6d0cd80e2804 Document additional error handling names available through PEP 293.
Walter Dörwald <walter@livinglogic.de>
parents: 26099
diff changeset
52 (replace with backslashed escape sequences (for encoding only)) as
6d0cd80e2804 Document additional error handling names available through PEP 293.
Walter Dörwald <walter@livinglogic.de>
parents: 26099
diff changeset
53 well as any other error handling name defined via
6d0cd80e2804 Document additional error handling names available through PEP 293.
Walter Dörwald <walter@livinglogic.de>
parents: 26099
diff changeset
54 \function{register_error()}.
12260
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
55
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
56 In case a search function cannot find a given encoding, it should
12274
559b25a79032 Make sure the \declaremodule uses the right name for the module!
Fred Drake <fdrake@acm.org>
parents: 12260
diff changeset
57 return \code{None}.
12260
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
58 \end{funcdesc}
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
59
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
60 \begin{funcdesc}{lookup}{encoding}
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
61 Looks up a codec tuple in the Python codec registry and returns the
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
62 function tuple as defined above.
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
63
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
64 Encodings are first looked up in the registry's cache. If not found,
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
65 the list of registered search functions is scanned. If no codecs tuple
12274
559b25a79032 Make sure the \declaremodule uses the right name for the module!
Fred Drake <fdrake@acm.org>
parents: 12260
diff changeset
66 is found, a \exception{LookupError} is raised. Otherwise, the codecs
559b25a79032 Make sure the \declaremodule uses the right name for the module!
Fred Drake <fdrake@acm.org>
parents: 12260
diff changeset
67 tuple is stored in the cache and returned to the caller.
12260
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
68 \end{funcdesc}
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
69
23112
Skip Montanaro <skip@pobox.com>
parents: 23087
diff changeset
70 To simplify access to the various codecs, the module provides these
19982
bf4b1b08a2c6 Docs and News item for the codecs.py additions.
Marc-André Lemburg <mal@egenix.com>
parents: 16127
diff changeset
71 additional functions which use \function{lookup()} for the codec
bf4b1b08a2c6 Docs and News item for the codecs.py additions.
Marc-André Lemburg <mal@egenix.com>
parents: 16127
diff changeset
72 lookup:
bf4b1b08a2c6 Docs and News item for the codecs.py additions.
Marc-André Lemburg <mal@egenix.com>
parents: 16127
diff changeset
73
bf4b1b08a2c6 Docs and News item for the codecs.py additions.
Marc-André Lemburg <mal@egenix.com>
parents: 16127
diff changeset
74 \begin{funcdesc}{getencoder}{encoding}
bf4b1b08a2c6 Docs and News item for the codecs.py additions.
Marc-André Lemburg <mal@egenix.com>
parents: 16127
diff changeset
75 Lookup up the codec for the given encoding and return its encoder
bf4b1b08a2c6 Docs and News item for the codecs.py additions.
Marc-André Lemburg <mal@egenix.com>
parents: 16127
diff changeset
76 function.
bf4b1b08a2c6 Docs and News item for the codecs.py additions.
Marc-André Lemburg <mal@egenix.com>
parents: 16127
diff changeset
77
bf4b1b08a2c6 Docs and News item for the codecs.py additions.
Marc-André Lemburg <mal@egenix.com>
parents: 16127
diff changeset
78 Raises a \exception{LookupError} in case the encoding cannot be found.
bf4b1b08a2c6 Docs and News item for the codecs.py additions.
Marc-André Lemburg <mal@egenix.com>
parents: 16127
diff changeset
79 \end{funcdesc}
bf4b1b08a2c6 Docs and News item for the codecs.py additions.
Marc-André Lemburg <mal@egenix.com>
parents: 16127
diff changeset
80
bf4b1b08a2c6 Docs and News item for the codecs.py additions.
Marc-André Lemburg <mal@egenix.com>
parents: 16127
diff changeset
81 \begin{funcdesc}{getdecoder}{encoding}
bf4b1b08a2c6 Docs and News item for the codecs.py additions.
Marc-André Lemburg <mal@egenix.com>
parents: 16127
diff changeset
82 Lookup up the codec for the given encoding and return its decoder
bf4b1b08a2c6 Docs and News item for the codecs.py additions.
Marc-André Lemburg <mal@egenix.com>
parents: 16127
diff changeset
83 function.
bf4b1b08a2c6 Docs and News item for the codecs.py additions.
Marc-André Lemburg <mal@egenix.com>
parents: 16127
diff changeset
84
bf4b1b08a2c6 Docs and News item for the codecs.py additions.
Marc-André Lemburg <mal@egenix.com>
parents: 16127
diff changeset
85 Raises a \exception{LookupError} in case the encoding cannot be found.
bf4b1b08a2c6 Docs and News item for the codecs.py additions.
Marc-André Lemburg <mal@egenix.com>
parents: 16127
diff changeset
86 \end{funcdesc}
bf4b1b08a2c6 Docs and News item for the codecs.py additions.
Marc-André Lemburg <mal@egenix.com>
parents: 16127
diff changeset
87
bf4b1b08a2c6 Docs and News item for the codecs.py additions.
Marc-André Lemburg <mal@egenix.com>
parents: 16127
diff changeset
88 \begin{funcdesc}{getreader}{encoding}
bf4b1b08a2c6 Docs and News item for the codecs.py additions.
Marc-André Lemburg <mal@egenix.com>
parents: 16127
diff changeset
89 Lookup up the codec for the given encoding and return its StreamReader
bf4b1b08a2c6 Docs and News item for the codecs.py additions.
Marc-André Lemburg <mal@egenix.com>
parents: 16127
diff changeset
90 class or factory function.
bf4b1b08a2c6 Docs and News item for the codecs.py additions.
Marc-André Lemburg <mal@egenix.com>
parents: 16127
diff changeset
91
bf4b1b08a2c6 Docs and News item for the codecs.py additions.
Marc-André Lemburg <mal@egenix.com>
parents: 16127
diff changeset
92 Raises a \exception{LookupError} in case the encoding cannot be found.
bf4b1b08a2c6 Docs and News item for the codecs.py additions.
Marc-André Lemburg <mal@egenix.com>
parents: 16127
diff changeset
93 \end{funcdesc}
bf4b1b08a2c6 Docs and News item for the codecs.py additions.
Marc-André Lemburg <mal@egenix.com>
parents: 16127
diff changeset
94
bf4b1b08a2c6 Docs and News item for the codecs.py additions.
Marc-André Lemburg <mal@egenix.com>
parents: 16127
diff changeset
95 \begin{funcdesc}{getwriter}{encoding}
bf4b1b08a2c6 Docs and News item for the codecs.py additions.
Marc-André Lemburg <mal@egenix.com>
parents: 16127
diff changeset
96 Lookup up the codec for the given encoding and return its StreamWriter
bf4b1b08a2c6 Docs and News item for the codecs.py additions.
Marc-André Lemburg <mal@egenix.com>
parents: 16127
diff changeset
97 class or factory function.
bf4b1b08a2c6 Docs and News item for the codecs.py additions.
Marc-André Lemburg <mal@egenix.com>
parents: 16127
diff changeset
98
bf4b1b08a2c6 Docs and News item for the codecs.py additions.
Marc-André Lemburg <mal@egenix.com>
parents: 16127
diff changeset
99 Raises a \exception{LookupError} in case the encoding cannot be found.
bf4b1b08a2c6 Docs and News item for the codecs.py additions.
Marc-André Lemburg <mal@egenix.com>
parents: 16127
diff changeset
100 \end{funcdesc}
bf4b1b08a2c6 Docs and News item for the codecs.py additions.
Marc-André Lemburg <mal@egenix.com>
parents: 16127
diff changeset
101
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 23658
diff changeset
102 \begin{funcdesc}{register_error}{name, error_handler}
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 23658
diff changeset
103 Register the error handling function \var{error_handler} under the
25172
5a17f0978326 Fix typo. Close SF Bug 606354.
Raymond Hettinger <python@rcn.com>
parents: 25091
diff changeset
104 name \var{name}. \var{error_handler} will be called during encoding
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 23658
diff changeset
105 and decoding in case of an error, when \var{name} is specified as the
27555
c267f28724dd Change the treatment of positions returned by PEP293
Walter Dörwald <walter@livinglogic.de>
parents: 26943
diff changeset
106 errors parameter.
c267f28724dd Change the treatment of positions returned by PEP293
Walter Dörwald <walter@livinglogic.de>
parents: 26943
diff changeset
107
c267f28724dd Change the treatment of positions returned by PEP293
Walter Dörwald <walter@livinglogic.de>
parents: 26943
diff changeset
108 For encoding \var{error_handler} will be called with a
c267f28724dd Change the treatment of positions returned by PEP293
Walter Dörwald <walter@livinglogic.de>
parents: 26943
diff changeset
109 \exception{UnicodeEncodeError} instance, which contains information about
c267f28724dd Change the treatment of positions returned by PEP293
Walter Dörwald <walter@livinglogic.de>
parents: 26943
diff changeset
110 the location of the error. The error handler must either raise this or
c267f28724dd Change the treatment of positions returned by PEP293
Walter Dörwald <walter@livinglogic.de>
parents: 26943
diff changeset
111 a different exception or return a tuple with a replacement for the
c267f28724dd Change the treatment of positions returned by PEP293
Walter Dörwald <walter@livinglogic.de>
parents: 26943
diff changeset
112 unencodable part of the input and a position where encoding should
c267f28724dd Change the treatment of positions returned by PEP293
Walter Dörwald <walter@livinglogic.de>
parents: 26943
diff changeset
113 continue. The encoder will encode the replacement and continue encoding
c267f28724dd Change the treatment of positions returned by PEP293
Walter Dörwald <walter@livinglogic.de>
parents: 26943
diff changeset
114 the original input at the specified position. Negative position values
c267f28724dd Change the treatment of positions returned by PEP293
Walter Dörwald <walter@livinglogic.de>
parents: 26943
diff changeset
115 will be treated as being relative to the end of the input string. If the
c267f28724dd Change the treatment of positions returned by PEP293
Walter Dörwald <walter@livinglogic.de>
parents: 26943
diff changeset
116 resulting position is out of bound an IndexError will be raised.
c267f28724dd Change the treatment of positions returned by PEP293
Walter Dörwald <walter@livinglogic.de>
parents: 26943
diff changeset
117
c267f28724dd Change the treatment of positions returned by PEP293
Walter Dörwald <walter@livinglogic.de>
parents: 26943
diff changeset
118 Decoding and translating works similar, except \exception{UnicodeDecodeError}
c267f28724dd Change the treatment of positions returned by PEP293
Walter Dörwald <walter@livinglogic.de>
parents: 26943
diff changeset
119 or \exception{UnicodeTranslateError} will be passed to the handler and
c267f28724dd Change the treatment of positions returned by PEP293
Walter Dörwald <walter@livinglogic.de>
parents: 26943
diff changeset
120 that the replacement from the error handler will be put into the output
c267f28724dd Change the treatment of positions returned by PEP293
Walter Dörwald <walter@livinglogic.de>
parents: 26943
diff changeset
121 directly.
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 23658
diff changeset
122 \end{funcdesc}
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 23658
diff changeset
123
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 23658
diff changeset
124 \begin{funcdesc}{lookup_error}{name}
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 23658
diff changeset
125 Return the error handler previously register under the name \var{name}.
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 23658
diff changeset
126
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 23658
diff changeset
127 Raises a \exception{LookupError} in case the handler cannot be found.
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 23658
diff changeset
128 \end{funcdesc}
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 23658
diff changeset
129
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 23658
diff changeset
130 \begin{funcdesc}{strict_errors}{exception}
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 23658
diff changeset
131 Implements the \code{strict} error handling.
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 23658
diff changeset
132 \end{funcdesc}
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 23658
diff changeset
133
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 23658
diff changeset
134 \begin{funcdesc}{replace_errors}{exception}
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 23658
diff changeset
135 Implements the \code{replace} error handling.
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 23658
diff changeset
136 \end{funcdesc}
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 23658
diff changeset
137
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 23658
diff changeset
138 \begin{funcdesc}{ignore_errors}{exception}
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 23658
diff changeset
139 Implements the \code{ignore} error handling.
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 23658
diff changeset
140 \end{funcdesc}
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 23658
diff changeset
141
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 23658
diff changeset
142 \begin{funcdesc}{xmlcharrefreplace_errors_errors}{exception}
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 23658
diff changeset
143 Implements the \code{xmlcharrefreplace} error handling.
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 23658
diff changeset
144 \end{funcdesc}
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 23658
diff changeset
145
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 23658
diff changeset
146 \begin{funcdesc}{backslashreplace_errors_errors}{exception}
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 23658
diff changeset
147 Implements the \code{backslashreplace} error handling.
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 23658
diff changeset
148 \end{funcdesc}
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 23658
diff changeset
149
25989
611af79c0e89 Move introductory sentence to where it belongs.
Walter Dörwald <walter@livinglogic.de>
parents: 25172
diff changeset
150 To simplify working with encoded files or stream, the module
611af79c0e89 Move introductory sentence to where it belongs.
Walter Dörwald <walter@livinglogic.de>
parents: 25172
diff changeset
151 also defines these utility functions:
611af79c0e89 Move introductory sentence to where it belongs.
Walter Dörwald <walter@livinglogic.de>
parents: 25172
diff changeset
152
13696
bb364d6f9f69 Fix small typos and markup consistency nits.
Fred Drake <fdrake@acm.org>
parents: 12274
diff changeset
153 \begin{funcdesc}{open}{filename, mode\optional{, encoding\optional{,
bb364d6f9f69 Fix small typos and markup consistency nits.
Fred Drake <fdrake@acm.org>
parents: 12274
diff changeset
154 errors\optional{, buffering}}}}
12260
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
155 Open an encoded file using the given \var{mode} and return
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
156 a wrapped version providing transparent encoding/decoding.
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
157
20580
376ee174ecd0 Use the \note and \warning macros where appropriate.
Fred Drake <fdrake@acm.org>
parents: 19982
diff changeset
158 \note{The wrapped version will only accept the object format
13696
bb364d6f9f69 Fix small typos and markup consistency nits.
Fred Drake <fdrake@acm.org>
parents: 12274
diff changeset
159 defined by the codecs, i.e.\ Unicode objects for most built-in
bb364d6f9f69 Fix small typos and markup consistency nits.
Fred Drake <fdrake@acm.org>
parents: 12274
diff changeset
160 codecs. Output is also codec-dependent and will usually be Unicode as
20580
376ee174ecd0 Use the \note and \warning macros where appropriate.
Fred Drake <fdrake@acm.org>
parents: 19982
diff changeset
161 well.}
12260
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
162
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
163 \var{encoding} specifies the encoding which is to be used for the
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
164 the file.
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
165
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
166 \var{errors} may be given to define the error handling. It defaults
13696
bb364d6f9f69 Fix small typos and markup consistency nits.
Fred Drake <fdrake@acm.org>
parents: 12274
diff changeset
167 to \code{'strict'} which causes a \exception{ValueError} to be raised
bb364d6f9f69 Fix small typos and markup consistency nits.
Fred Drake <fdrake@acm.org>
parents: 12274
diff changeset
168 in case an encoding error occurs.
12260
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
169
12274
559b25a79032 Make sure the \declaremodule uses the right name for the module!
Fred Drake <fdrake@acm.org>
parents: 12260
diff changeset
170 \var{buffering} has the same meaning as for the built-in
559b25a79032 Make sure the \declaremodule uses the right name for the module!
Fred Drake <fdrake@acm.org>
parents: 12260
diff changeset
171 \function{open()} function. It defaults to line buffered.
12260
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
172 \end{funcdesc}
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
173
13696
bb364d6f9f69 Fix small typos and markup consistency nits.
Fred Drake <fdrake@acm.org>
parents: 12274
diff changeset
174 \begin{funcdesc}{EncodedFile}{file, input\optional{,
bb364d6f9f69 Fix small typos and markup consistency nits.
Fred Drake <fdrake@acm.org>
parents: 12274
diff changeset
175 output\optional{, errors}}}
12260
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
176 Return a wrapped version of file which provides transparent
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
177 encoding translation.
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
178
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
179 Strings written to the wrapped file are interpreted according to the
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
180 given \var{input} encoding and then written to the original file as
13696
bb364d6f9f69 Fix small typos and markup consistency nits.
Fred Drake <fdrake@acm.org>
parents: 12274
diff changeset
181 strings using the \var{output} encoding. The intermediate encoding will
12260
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
182 usually be Unicode but depends on the specified codecs.
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
183
13696
bb364d6f9f69 Fix small typos and markup consistency nits.
Fred Drake <fdrake@acm.org>
parents: 12274
diff changeset
184 If \var{output} is not given, it defaults to \var{input}.
12260
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
185
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
186 \var{errors} may be given to define the error handling. It defaults to
13696
bb364d6f9f69 Fix small typos and markup consistency nits.
Fred Drake <fdrake@acm.org>
parents: 12274
diff changeset
187 \code{'strict'}, which causes \exception{ValueError} to be raised in case
12260
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
188 an encoding error occurs.
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
189 \end{funcdesc}
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
190
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
191 The module also provides the following constants which are useful
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
192 for reading and writing to platform dependent files:
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
193
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
194 \begin{datadesc}{BOM}
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
195 \dataline{BOM_BE}
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
196 \dataline{BOM_LE}
23658
94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE,
Walter Dörwald <walter@livinglogic.de>
parents: 23112
diff changeset
197 \dataline{BOM_UTF8}
94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE,
Walter Dörwald <walter@livinglogic.de>
parents: 23112
diff changeset
198 \dataline{BOM_UTF16}
94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE,
Walter Dörwald <walter@livinglogic.de>
parents: 23112
diff changeset
199 \dataline{BOM_UTF16_BE}
94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE,
Walter Dörwald <walter@livinglogic.de>
parents: 23112
diff changeset
200 \dataline{BOM_UTF16_LE}
94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE,
Walter Dörwald <walter@livinglogic.de>
parents: 23112
diff changeset
201 \dataline{BOM_UTF32}
94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE,
Walter Dörwald <walter@livinglogic.de>
parents: 23112
diff changeset
202 \dataline{BOM_UTF32_BE}
94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE,
Walter Dörwald <walter@livinglogic.de>
parents: 23112
diff changeset
203 \dataline{BOM_UTF32_LE}
94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE,
Walter Dörwald <walter@livinglogic.de>
parents: 23112
diff changeset
204 These constants define various encodings of the Unicode byte order mark
94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE,
Walter Dörwald <walter@livinglogic.de>
parents: 23112
diff changeset
205 (BOM) used in UTF-16 and UTF-32 data streams to indicate the byte order
94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE,
Walter Dörwald <walter@livinglogic.de>
parents: 23112
diff changeset
206 used in the stream or file and in UTF-8 as a Unicode signature.
94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE,
Walter Dörwald <walter@livinglogic.de>
parents: 23112
diff changeset
207 \constant{BOM_UTF16} is either \constant{BOM_UTF16_BE} or
94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE,
Walter Dörwald <walter@livinglogic.de>
parents: 23112
diff changeset
208 \constant{BOM_UTF16_LE} depending on the platform's native byte order,
94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE,
Walter Dörwald <walter@livinglogic.de>
parents: 23112
diff changeset
209 \constant{BOM} is an alias for \constant{BOM_UTF16}, \constant{BOM_LE}
94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE,
Walter Dörwald <walter@livinglogic.de>
parents: 23112
diff changeset
210 for \constant{BOM_UTF16_LE} and \constant{BOM_BE} for \constant{BOM_UTF16_BE}.
94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE,
Walter Dörwald <walter@livinglogic.de>
parents: 23112
diff changeset
211 The others represent the BOM in UTF-8 and UTF-32 encodings.
12260
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
212 \end{datadesc}
8c0f863ba50c Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
diff changeset
213
16127
6f1fd8500398 Added link to the "Python Codecs" project at SourceForge.
Fred Drake <fdrake@acm.org>
parents: 15186
diff changeset
214
6f1fd8500398 Added link to the "Python Codecs" project at SourceForge.
Fred Drake <fdrake@acm.org>
parents: 15186
diff changeset
215 \begin{seealso}
6f1fd8500398 Added link to the "Python Codecs" project at SourceForge.
Fred Drake <fdrake@acm.org>
parents: 15186
diff changeset
216 \seeurl{http://sourceforge.net/projects/python-codecs/}{A
6f1fd8500398 Added link to the "Python Codecs" project at SourceForge.
Fred Drake <fdrake@acm.org>
parents: 15186
diff changeset
217 SourceForge project working on additional support for Asian
6f1fd8500398 Added link to the "Python Codecs" project at SourceForge.
Fred Drake <fdrake@acm.org>
parents: 15186
diff changeset
218 codecs for use with Python. They are in the early stages of
6f1fd8500398 Added link to the "Python Codecs" project at SourceForge.
Fred Drake <fdrake@acm.org>
parents: 15186
diff changeset
219 development at the time of this writing --- look in their
6f1fd8500398 Added link to the "Python Codecs" project at SourceForge.
Fred Drake <fdrake@acm.org>
parents: 15186
diff changeset
220 FTP area for downloadable files.}
6f1fd8500398 Added link to the "Python Codecs" project at SourceForge.
Fred Drake <fdrake@acm.org>
parents: 15186
diff changeset
221 \end{seealso}
6f1fd8500398 Added link to the "Python Codecs" project at SourceForge.
Fred Drake <fdrake@acm.org>
parents: 15186
diff changeset
222
6f1fd8500398 Added link to the "Python Codecs" project at SourceForge.
Fred Drake <fdrake@acm.org>
parents: 15186
diff changeset
223
15186
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
224 \subsection{Codec Base Classes}
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
225
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
226 The \module{codecs} defines a set of base classes which define the
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
227 interface and can also be used to easily write you own codecs for use
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
228 in Python.
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
229
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
230 Each codec has to define four interfaces to make it usable as codec in
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
231 Python: stateless encoder, stateless decoder, stream reader and stream
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
232 writer. The stream reader and writers typically reuse the stateless
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
233 encoder/decoder to implement the file protocols.
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
234
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
235 The \class{Codec} class defines the interface for stateless
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
236 encoders/decoders.
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
237
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
238 To simplify and standardize error handling, the \method{encode()} and
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
239 \method{decode()} methods may implement different error handling
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
240 schemes by providing the \var{errors} string argument. The following
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
241 string values are defined and implemented by all standard Python
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
242 codecs:
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
243
16127
6f1fd8500398 Added link to the "Python Codecs" project at SourceForge.
Fred Drake <fdrake@acm.org>
parents: 15186
diff changeset
244 \begin{tableii}{l|l}{code}{Value}{Meaning}
26099
135f08b73bd5 Add documentation for the PEP 293 functionality:
Walter Dörwald <walter@livinglogic.de>
parents: 25989
diff changeset
245 \lineii{'strict'}{Raise \exception{UnicodeError} (or a subclass);
16127
6f1fd8500398 Added link to the "Python Codecs" project at SourceForge.
Fred Drake <fdrake@acm.org>
parents: 15186
diff changeset
246 this is the default.}
6f1fd8500398 Added link to the "Python Codecs" project at SourceForge.
Fred Drake <fdrake@acm.org>
parents: 15186
diff changeset
247 \lineii{'ignore'}{Ignore the character and continue with the next.}
6f1fd8500398 Added link to the "Python Codecs" project at SourceForge.
Fred Drake <fdrake@acm.org>
parents: 15186
diff changeset
248 \lineii{'replace'}{Replace with a suitable replacement character;
6f1fd8500398 Added link to the "Python Codecs" project at SourceForge.
Fred Drake <fdrake@acm.org>
parents: 15186
diff changeset
249 Python will use the official U+FFFD REPLACEMENT
26099
135f08b73bd5 Add documentation for the PEP 293 functionality:
Walter Dörwald <walter@livinglogic.de>
parents: 25989
diff changeset
250 CHARACTER for the built-in Unicode codecs on
135f08b73bd5 Add documentation for the PEP 293 functionality:
Walter Dörwald <walter@livinglogic.de>
parents: 25989
diff changeset
251 decoding and '?' on encoding.}
135f08b73bd5 Add documentation for the PEP 293 functionality:
Walter Dörwald <walter@livinglogic.de>
parents: 25989
diff changeset
252 \lineii{'xmlcharrefreplace'}{Replace with the appropriate XML
135f08b73bd5 Add documentation for the PEP 293 functionality:
Walter Dörwald <walter@livinglogic.de>
parents: 25989
diff changeset
253 character reference (only for encoding).}
135f08b73bd5 Add documentation for the PEP 293 functionality:
Walter Dörwald <walter@livinglogic.de>
parents: 25989
diff changeset
254 \lineii{'backslashreplace'}{Replace with backslashed escape sequences
135f08b73bd5 Add documentation for the PEP 293 functionality:
Walter Dörwald <walter@livinglogic.de>
parents: 25989
diff changeset
255 (only for encoding).}
16127
6f1fd8500398 Added link to the "Python Codecs" project at SourceForge.
Fred Drake <fdrake@acm.org>
parents: 15186
diff changeset
256 \end{tableii}
15186
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
257
26099
135f08b73bd5 Add documentation for the PEP 293 functionality:
Walter Dörwald <walter@livinglogic.de>
parents: 25989
diff changeset
258 The set of allowed values can be extended via \method{register_error}.
135f08b73bd5 Add documentation for the PEP 293 functionality:
Walter Dörwald <walter@livinglogic.de>
parents: 25989
diff changeset
259
15186
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
260
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
261 \subsubsection{Codec Objects \label{codec-objects}}
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
262
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
263 The \class{Codec} class defines these methods which also define the
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
264 function interfaces of the stateless encoder and decoder:
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
265
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
266 \begin{methoddesc}{encode}{input\optional{, errors}}
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
267 Encodes the object \var{input} and returns a tuple (output object,
23087
09db3f5d0bd0 added small clarification to the descriptions of encode() and decode()
Skip Montanaro <skip@pobox.com>
parents: 20580
diff changeset
268 length consumed). While codecs are not restricted to use with Unicode, in
09db3f5d0bd0 added small clarification to the descriptions of encode() and decode()
Skip Montanaro <skip@pobox.com>
parents: 20580
diff changeset
269 a Unicode context, encoding converts a Unicode object to a plain string
09db3f5d0bd0 added small clarification to the descriptions of encode() and decode()
Skip Montanaro <skip@pobox.com>
parents: 20580
diff changeset
270 using a particular character set encoding (e.g., \code{cp1252} or
09db3f5d0bd0 added small clarification to the descriptions of encode() and decode()
Skip Montanaro <skip@pobox.com>
parents: 20580
diff changeset
271 \code{iso-8859-1}).
15186
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
272
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
273 \var{errors} defines the error handling to apply. It defaults to
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
274 \code{'strict'} handling.
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
275
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
276 The method may not store state in the \class{Codec} instance. Use
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
277 \class{StreamCodec} for codecs which have to keep state in order to
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
278 make encoding/decoding efficient.
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
279
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
280 The encoder must be able to handle zero length input and return an
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
281 empty object of the output object type in this situation.
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
282 \end{methoddesc}
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
283
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
284 \begin{methoddesc}{decode}{input\optional{, errors}}
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
285 Decodes the object \var{input} and returns a tuple (output object,
23087
09db3f5d0bd0 added small clarification to the descriptions of encode() and decode()
Skip Montanaro <skip@pobox.com>
parents: 20580
diff changeset
286 length consumed). In a Unicode context, decoding converts a plain string
09db3f5d0bd0 added small clarification to the descriptions of encode() and decode()
Skip Montanaro <skip@pobox.com>
parents: 20580
diff changeset
287 encoded using a particular character set encoding to a Unicode object.
15186
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
288
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
289 \var{input} must be an object which provides the \code{bf_getreadbuf}
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
290 buffer slot. Python strings, buffer objects and memory mapped files
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
291 are examples of objects providing this slot.
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
292
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
293 \var{errors} defines the error handling to apply. It defaults to
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
294 \code{'strict'} handling.
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
295
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
296 The method may not store state in the \class{Codec} instance. Use
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
297 \class{StreamCodec} for codecs which have to keep state in order to
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
298 make encoding/decoding efficient.
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
299
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
300 The decoder must be able to handle zero length input and return an
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
301 empty object of the output object type in this situation.
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
302 \end{methoddesc}
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
303
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
304 The \class{StreamWriter} and \class{StreamReader} classes provide
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
305 generic working interfaces which can be used to implement new
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
306 encodings submodules very easily. See \module{encodings.utf_8} for an
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
307 example on how this is done.
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
308
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
309
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
310 \subsubsection{StreamWriter Objects \label{stream-writer-objects}}
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
311
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
312 The \class{StreamWriter} class is a subclass of \class{Codec} and
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
313 defines the following methods which every stream writer must define in
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
314 order to be compatible to the Python codec registry.
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
315
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
316 \begin{classdesc}{StreamWriter}{stream\optional{, errors}}
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
317 Constructor for a \class{StreamWriter} instance.
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
318
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
319 All stream writers must provide this constructor interface. They are
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
320 free to add additional keyword arguments, but only the ones defined
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
321 here are used by the Python codec registry.
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
322
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
323 \var{stream} must be a file-like object open for writing (binary)
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
324 data.
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
325
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
326 The \class{StreamWriter} may implement different error handling
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
327 schemes by providing the \var{errors} keyword argument. These
26099
135f08b73bd5 Add documentation for the PEP 293 functionality:
Walter Dörwald <walter@livinglogic.de>
parents: 25989
diff changeset
328 parameters are predefined:
15186
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
329
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
330 \begin{itemize}
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
331 \item \code{'strict'} Raise \exception{ValueError} (or a subclass);
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
332 this is the default.
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
333 \item \code{'ignore'} Ignore the character and continue with the next.
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
334 \item \code{'replace'} Replace with a suitable replacement character
26099
135f08b73bd5 Add documentation for the PEP 293 functionality:
Walter Dörwald <walter@livinglogic.de>
parents: 25989
diff changeset
335 \item \code{'xmlcharrefreplace'} Replace with the appropriate XML
135f08b73bd5 Add documentation for the PEP 293 functionality:
Walter Dörwald <walter@livinglogic.de>
parents: 25989
diff changeset
336 character reference
135f08b73bd5 Add documentation for the PEP 293 functionality:
Walter Dörwald <walter@livinglogic.de>
parents: 25989
diff changeset
337 \item \code{'backslashreplace'} Replace with backslashed escape sequences.
15186
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
338 \end{itemize}
26099
135f08b73bd5 Add documentation for the PEP 293 functionality:
Walter Dörwald <walter@livinglogic.de>
parents: 25989
diff changeset
339
135f08b73bd5 Add documentation for the PEP 293 functionality:
Walter Dörwald <walter@livinglogic.de>
parents: 25989
diff changeset
340 The \var{errors} argument will be assigned to an attribute of the
135f08b73bd5 Add documentation for the PEP 293 functionality:
Walter Dörwald <walter@livinglogic.de>
parents: 25989
diff changeset
341 same name. Assigning to this attribute makes it possible to switch
135f08b73bd5 Add documentation for the PEP 293 functionality:
Walter Dörwald <walter@livinglogic.de>
parents: 25989
diff changeset
342 between different error handling strategies during the lifetime
135f08b73bd5 Add documentation for the PEP 293 functionality:
Walter Dörwald <walter@livinglogic.de>
parents: 25989
diff changeset
343 of the \class{StreamWriter} object.
135f08b73bd5 Add documentation for the PEP 293 functionality:
Walter Dörwald <walter@livinglogic.de>
parents: 25989
diff changeset
344
135f08b73bd5 Add documentation for the PEP 293 functionality:
Walter Dörwald <walter@livinglogic.de>
parents: 25989
diff changeset
345 The set of allowed values for the \var{errors} argument can
135f08b73bd5 Add documentation for the PEP 293 functionality:
Walter Dörwald <walter@livinglogic.de>
parents: 25989
diff changeset
346 be extended with \function{register_error()}.
15186
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
347 \end{classdesc}
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
348
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
349 \begin{methoddesc}{write}{object}
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
350 Writes the object's contents encoded to the stream.
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
351 \end{methoddesc}
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
352
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
353 \begin{methoddesc}{writelines}{list}
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
354 Writes the concatenated list of strings to the stream (possibly by
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
355 reusing the \method{write()} method).
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
356 \end{methoddesc}
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
357
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
358 \begin{methoddesc}{reset}{}
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
359 Flushes and resets the codec buffers used for keeping state.
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
360
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
361 Calling this method should ensure that the data on the output is put
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
362 into a clean state, that allows appending of new fresh data without
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
363 having to rescan the whole stream to recover state.
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
364 \end{methoddesc}
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
365
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
366 In addition to the above methods, the \class{StreamWriter} must also
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
367 inherit all other methods and attribute from the underlying stream.
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
368
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
369
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
370 \subsubsection{StreamReader Objects \label{stream-reader-objects}}
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
371
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
372 The \class{StreamReader} class is a subclass of \class{Codec} and
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
373 defines the following methods which every stream reader must define in
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
374 order to be compatible to the Python codec registry.
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
375
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
376 \begin{classdesc}{StreamReader}{stream\optional{, errors}}
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
377 Constructor for a \class{StreamReader} instance.
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
378
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
379 All stream readers must provide this constructor interface. They are
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
380 free to add additional keyword arguments, but only the ones defined
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
381 here are used by the Python codec registry.
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
382
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
383 \var{stream} must be a file-like object open for reading (binary)
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
384 data.
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
385
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
386 The \class{StreamReader} may implement different error handling
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
387 schemes by providing the \var{errors} keyword argument. These
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
388 parameters are defined:
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
389
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
390 \begin{itemize}
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
391 \item \code{'strict'} Raise \exception{ValueError} (or a subclass);
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
392 this is the default.
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
393 \item \code{'ignore'} Ignore the character and continue with the next.
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
394 \item \code{'replace'} Replace with a suitable replacement character.
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
395 \end{itemize}
26099
135f08b73bd5 Add documentation for the PEP 293 functionality:
Walter Dörwald <walter@livinglogic.de>
parents: 25989
diff changeset
396
135f08b73bd5 Add documentation for the PEP 293 functionality:
Walter Dörwald <walter@livinglogic.de>
parents: 25989
diff changeset
397 The \var{errors} argument will be assigned to an attribute of the
135f08b73bd5 Add documentation for the PEP 293 functionality:
Walter Dörwald <walter@livinglogic.de>
parents: 25989
diff changeset
398 same name. Assigning to this attribute makes it possible to switch
135f08b73bd5 Add documentation for the PEP 293 functionality:
Walter Dörwald <walter@livinglogic.de>
parents: 25989
diff changeset
399 between different error handling strategies during the lifetime
135f08b73bd5 Add documentation for the PEP 293 functionality:
Walter Dörwald <walter@livinglogic.de>
parents: 25989
diff changeset
400 of the \class{StreamReader} object.
135f08b73bd5 Add documentation for the PEP 293 functionality:
Walter Dörwald <walter@livinglogic.de>
parents: 25989
diff changeset
401
135f08b73bd5 Add documentation for the PEP 293 functionality:
Walter Dörwald <walter@livinglogic.de>
parents: 25989
diff changeset
402 The set of allowed values for the \var{errors} argument can
135f08b73bd5 Add documentation for the PEP 293 functionality:
Walter Dörwald <walter@livinglogic.de>
parents: 25989
diff changeset
403 be extended with \function{register_error()}.
15186
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
404 \end{classdesc}
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
405
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
406 \begin{methoddesc}{read}{\optional{size}}
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
407 Decodes data from the stream and returns the resulting object.
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
408
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
409 \var{size} indicates the approximate maximum number of bytes to read
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
410 from the stream for decoding purposes. The decoder can modify this
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
411 setting as appropriate. The default value -1 indicates to read and
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
412 decode as much as possible. \var{size} is intended to prevent having
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
413 to decode huge files in one step.
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
414
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
415 The method should use a greedy read strategy meaning that it should
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
416 read as much data as is allowed within the definition of the encoding
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
417 and the given size, e.g. if optional encoding endings or state
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
418 markers are available on the stream, these should be read too.
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
419 \end{methoddesc}
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
420
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
421 \begin{methoddesc}{readline}{[size]}
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
422 Read one line from the input stream and return the
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
423 decoded data.
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
424
20580
376ee174ecd0 Use the \note and \warning macros where appropriate.
Fred Drake <fdrake@acm.org>
parents: 19982
diff changeset
425 Unlike the \method{readlines()} method, this method inherits
15186
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
426 the line breaking knowledge from the underlying stream's
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
427 \method{readline()} method -- there is currently no support for line
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
428 breaking using the codec decoder due to lack of line buffering.
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
429 Sublcasses should however, if possible, try to implement this method
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
430 using their own knowledge of line breaking.
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
431
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
432 \var{size}, if given, is passed as size argument to the stream's
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
433 \method{readline()} method.
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
434 \end{methoddesc}
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
435
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
436 \begin{methoddesc}{readlines}{[sizehint]}
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
437 Read all lines available on the input stream and return them as list
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
438 of lines.
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
439
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
440 Line breaks are implemented using the codec's decoder method and are
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
441 included in the list entries.
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
442
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
443 \var{sizehint}, if given, is passed as \var{size} argument to the
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
444 stream's \method{read()} method.
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
445 \end{methoddesc}
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
446
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
447 \begin{methoddesc}{reset}{}
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
448 Resets the codec buffers used for keeping state.
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
449
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
450 Note that no stream repositioning should take place. This method is
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
451 primarily intended to be able to recover from decoding errors.
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
452 \end{methoddesc}
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
453
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
454 In addition to the above methods, the \class{StreamReader} must also
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
455 inherit all other methods and attribute from the underlying stream.
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
456
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
457 The next two base classes are included for convenience. They are not
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
458 needed by the codec registry, but may provide useful in practice.
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
459
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
460
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
461 \subsubsection{StreamReaderWriter Objects \label{stream-reader-writer}}
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
462
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
463 The \class{StreamReaderWriter} allows wrapping streams which work in
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
464 both read and write modes.
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
465
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
466 The design is such that one can use the factory functions returned by
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
467 the \function{lookup()} function to construct the instance.
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
468
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
469 \begin{classdesc}{StreamReaderWriter}{stream, Reader, Writer, errors}
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
470 Creates a \class{StreamReaderWriter} instance.
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
471 \var{stream} must be a file-like object.
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
472 \var{Reader} and \var{Writer} must be factory functions or classes
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
473 providing the \class{StreamReader} and \class{StreamWriter} interface
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
474 resp.
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
475 Error handling is done in the same way as defined for the
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
476 stream readers and writers.
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
477 \end{classdesc}
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
478
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
479 \class{StreamReaderWriter} instances define the combined interfaces of
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
480 \class{StreamReader} and \class{StreamWriter} classes. They inherit
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
481 all other methods and attribute from the underlying stream.
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
482
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
483
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
484 \subsubsection{StreamRecoder Objects \label{stream-recoder-objects}}
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
485
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
486 The \class{StreamRecoder} provide a frontend - backend view of
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
487 encoding data which is sometimes useful when dealing with different
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
488 encoding environments.
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
489
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
490 The design is such that one can use the factory functions returned by
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
491 the \function{lookup()} function to construct the instance.
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
492
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
493 \begin{classdesc}{StreamRecoder}{stream, encode, decode,
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
494 Reader, Writer, errors}
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
495 Creates a \class{StreamRecoder} instance which implements a two-way
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
496 conversion: \var{encode} and \var{decode} work on the frontend (the
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
497 input to \method{read()} and output of \method{write()}) while
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
498 \var{Reader} and \var{Writer} work on the backend (reading and
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
499 writing to the stream).
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
500
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
501 You can use these objects to do transparent direct recodings from
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
502 e.g.\ Latin-1 to UTF-8 and back.
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
503
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
504 \var{stream} must be a file-like object.
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
505
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
506 \var{encode}, \var{decode} must adhere to the \class{Codec}
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
507 interface, \var{Reader}, \var{Writer} must be factory functions or
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
508 classes providing objects of the the \class{StreamReader} and
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
509 \class{StreamWriter} interface respectively.
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
510
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
511 \var{encode} and \var{decode} are needed for the frontend
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
512 translation, \var{Reader} and \var{Writer} for the backend
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
513 translation. The intermediate format used is determined by the two
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
514 sets of codecs, e.g. the Unicode codecs will use Unicode as
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
515 intermediate encoding.
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
516
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
517 Error handling is done in the same way as defined for the
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
518 stream readers and writers.
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
519 \end{classdesc}
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
520
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
521 \class{StreamRecoder} instances define the combined interfaces of
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
522 \class{StreamReader} and \class{StreamWriter} classes. They inherit
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
523 all other methods and attribute from the underlying stream.
b2892cbc63db Marc-Andre Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 13696
diff changeset
524
26943
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
525 \subsection{Standard Encodings}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
526
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
527 Python comes with a number of codecs builtin, either implemented as C
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
528 functions, or with dictionaries as mapping tables. The following table
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
529 lists the codecs by name, together with a few common aliases, and the
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
530 languages for which the encoding is likely used. Neither the list of
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
531 aliases nor the list of languages is meant to be exhaustive. Notice
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
532 that spelling alternatives that only differ in case or use a hyphen
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
533 instead of an underscore are also valid aliases.
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
534
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
535 Many of the character sets support the same languages. They vary in
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
536 individual characters (e.g. whether the EURO SIGN is supported or
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
537 not), and in the assignment of characters to code positions. For the
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
538 European languages in particular, the following variants typically
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
539 exist:
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
540
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
541 \begin{itemize}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
542 \item an ISO 8859 codeset
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
543 \item a Microsoft Windows code page, which is typically derived from
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
544 a 8859 codeset, but replaces control characters with additional
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
545 graphic characters
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
546 \item an IBM EBCDIC code page
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
547 \item an IBM PC code page, which is ASCII compatible
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
548 \end{itemize}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
549
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
550 \begin{longtableiii}{l|l|l}{textrm}{Codec}{Aliases}{Languages}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
551
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
552 \lineiii{ascii}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
553 {646, us-ascii}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
554 {English}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
555
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
556 \lineiii{cp037}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
557 {IBM037, IBM039}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
558 {English}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
559
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
560 \lineiii{cp424}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
561 {EBCDIC-CP-HE, IBM424}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
562 {Hebrew}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
563
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
564 \lineiii{cp437}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
565 {437, IBM437}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
566 {English}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
567
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
568 \lineiii{cp500}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
569 {EBCDIC-CP-BE, EBCDIC-CP-CH, IBM500}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
570 {Western Europe}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
571
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
572 \lineiii{cp737}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
573 {}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
574 {Greek}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
575
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
576 \lineiii{cp775}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
577 {IBM775}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
578 {Baltic languages}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
579
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
580 \lineiii{cp850}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
581 {850, IBM850}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
582 {Western Europe}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
583
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
584 \lineiii{cp852}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
585 {852, IBM852}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
586 {Central and Eastern Europe}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
587
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
588 \lineiii{cp855}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
589 {855, IBM855}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
590 {Bulgarian, Byelorussian, Macedonian, Russian, Serbian}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
591
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
592 \lineiii{cp856}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
593 {}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
594 {Hebrew}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
595
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
596 \lineiii{cp857}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
597 {857, IBM857}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
598 {Turkish}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
599
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
600 \lineiii{cp860}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
601 {860, IBM860}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
602 {Portuguese}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
603
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
604 \lineiii{cp861}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
605 {861, CP-IS, IBM861}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
606 {Icelandic}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
607
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
608 \lineiii{cp862}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
609 {862, IBM862}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
610 {Hebrew}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
611
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
612 \lineiii{cp863}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
613 {863, IBM863}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
614 {Canadian}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
615
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
616 \lineiii{cp864}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
617 {IBM864}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
618 {Arabic}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
619
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
620 \lineiii{cp865}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
621 {865, IBM865}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
622 {Danish, Norwegian}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
623
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
624 \lineiii{cp869}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
625 {869, CP-GR, IBM869}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
626 {Greek}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
627
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
628 \lineiii{cp874}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
629 {}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
630 {Thai}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
631
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
632 \lineiii{cp875}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
633 {}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
634 {Greek}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
635
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
636 \lineiii{cp1006}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
637 {}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
638 {Urdu}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
639
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
640 \lineiii{cp1026}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
641 {ibm1026}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
642 {Turkish}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
643
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
644 \lineiii{cp1140}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
645 {ibm1140}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
646 {Western Europe}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
647
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
648 \lineiii{cp1250}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
649 {windows-1250}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
650 {Central and Eastern Europe}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
651
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
652 \lineiii{cp1251}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
653 {windows-1251}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
654 {Bulgarian, Byelorussian, Macedonian, Russian, Serbian}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
655
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
656 \lineiii{cp1252}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
657 {windows-1252}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
658 {Western Europe}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
659
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
660 \lineiii{cp1253}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
661 {windows-1253}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
662 {Greek}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
663
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
664 \lineiii{cp1254}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
665 {windows-1254}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
666 {Turkish}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
667
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
668 \lineiii{cp1255}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
669 {windows-1255}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
670 {Hebrew}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
671
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
672 \lineiii{cp1256}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
673 {windows1256}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
674 {Arabic}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
675
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
676 \lineiii{cp1257}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
677 {windows-1257}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
678 {Baltic languages}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
679
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
680 \lineiii{cp1258}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
681 {windows-1258}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
682 {Vietnamese}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
683
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
684 \lineiii{latin_1}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
685 {iso-8859-1, iso8859-1, 8859, cp819, latin, latin1, L1}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
686 {West Europe}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
687
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
688 \lineiii{iso8859_2}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
689 {iso-8859-2, latin2, L2}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
690 {Central and Eastern Europe}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
691
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
692 \lineiii{iso8859_3}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
693 {iso-8859-3, latin3, L3}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
694 {Esperanto, Maltese}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
695
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
696 \lineiii{iso8859_4}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
697 {iso-8859-4, latin4, L4}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
698 {Baltic languagues}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
699
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
700 \lineiii{iso8859_5}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
701 {iso-8859-5, cyrillic}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
702 {Bulgarian, Byelorussian, Macedonian, Russian, Serbian}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
703
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
704 \lineiii{iso8859_6}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
705 {iso-8859-6, arabic}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
706 {Arabic}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
707
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
708 \lineiii{iso8859_7}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
709 {iso-8859-7, greek, greek8}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
710 {Greek}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
711
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
712 \lineiii{iso8859_8}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
713 {iso-8859-8, hebrew}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
714 {Hebrew}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
715
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
716 \lineiii{iso8859_9}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
717 {iso-8859-9, latin5, L5}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
718 {Turkish}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
719
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
720 \lineiii{iso8859_10}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
721 {iso-8859-10, latin6, L6}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
722 {Nordic languages}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
723
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
724 \lineiii{iso8859_13}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
725 {iso-8859-13}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
726 {Baltic languages}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
727
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
728 \lineiii{iso8859_14}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
729 {iso-8859-14, latin8, L8}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
730 {Celtic languages}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
731
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
732 \lineiii{iso8859_15}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
733 {iso-8859-15}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
734 {Western Europe}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
735
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
736 \lineiii{koi8_r}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
737 {}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
738 {Russian}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
739
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
740 \lineiii{koi8_u}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
741 {}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
742 {Ukrainian}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
743
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
744 \lineiii{mac_cyrillic}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
745 {maccyrillic}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
746 {Bulgarian, Byelorussian, Macedonian, Russian, Serbian}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
747
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
748 \lineiii{mac_greek}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
749 {macgreek}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
750 {Greek}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
751
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
752 \lineiii{mac_iceland}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
753 {maciceland}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
754 {Icelandic}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
755
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
756 \lineiii{mac_latin2}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
757 {maclatin2, maccentraleurope}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
758 {Central and Eastern Europe}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
759
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
760 \lineiii{mac_roman}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
761 {macroman}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
762 {Western Europe}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
763
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
764 \lineiii{mac_turkish}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
765 {macturkish}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
766 {Turkish}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
767
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
768 \lineiii{utf_16}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
769 {U16, utf16}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
770 {all languages}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
771
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
772 \lineiii{utf_16_be}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
773 {UTF-16BE}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
774 {all languages (BMP only)}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
775
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
776 \lineiii{utf_16_le}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
777 {UTF-16LE}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
778 {all languages (BMP only)}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
779
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
780 \lineiii{utf_7}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
781 {U7}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
782 {all languages}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
783
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
784 \lineiii{utf_8}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
785 {U8, UTF, utf8}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
786 {all languages}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
787
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
788 \end{longtableiii}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
789
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
790 A number of codecs are specific to Python, so their codec names have
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
791 no meaning outside Python. Some of them don't convert from Unicode
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
792 strings to byte strings, but instead use the property of the Python
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
793 codecs machinery that any bijective function with one argument can be
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
794 considered as an encoding.
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
795
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
796 For the codecs listed below, the result in the ``encoding'' direction
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
797 is always a byte string. The result of the ``decoding'' direction is
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
798 listed as operand type in the table.
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
799
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
800 \begin{tableiv}{l|l|l|l}{textrm}{Codec}{Aliases}{Operand type}{Purpose}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
801
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
802 \lineiv{base64_codec}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
803 {base64, base-64}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
804 {byte string}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
805 {Convert operand to MIME base64}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
806
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
807 \lineiv{hex_codec}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
808 {hex}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
809 {byte string}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
810 {Convert operand to hexadecimal representation, with two digits per byte}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
811
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
812 \lineiv{mbcs}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
813 {dbcs}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
814 {Unicode string}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
815 {Windows only: Encode operand according to the ANSI codepage (CP_ACP)}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
816
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
817 \lineiv{palmos}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
818 {}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
819 {Unicode string}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
820 {Encoding of PalmOS 3.5}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
821
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
822 \lineiv{quopri_codec}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
823 {quopri, quoted-printable, quotedprintable}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
824 {byte string}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
825 {Convert operand to MIME quoted printable}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
826
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
827 \lineiv{raw_unicode_escape}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
828 {}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
829 {Unicode string}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
830 {Produce a string that is suitable as raw Unicode literal in Python source code}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
831
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
832 \lineiv{rot_13}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
833 {rot13}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
834 {byte string}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
835 {Returns the Caesar-cypher encryption of the operand}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
836
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
837 \lineiv{string_escape}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
838 {}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
839 {byte string}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
840 {Produce a string that is suitable as string literal in Python source code}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
841
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
842 \lineiv{undefined}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
843 {}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
844 {any}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
845 {Raise an exception for all conversion. Can be used as the system encoding if no automatic coercion between byte and Unicode strings is desired.}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
846
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
847 \lineiv{unicode_escape}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
848 {}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
849 {Unicode string}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
850 {Produce a string that is suitable as Unicode literal in Python source code}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
851
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
852 \lineiv{unicode_internal}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
853 {}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
854 {Unicode string}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
855 {Return the internal represenation of the operand}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
856
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
857 \lineiv{uu_codec}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
858 {uu}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
859 {byte string}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
860 {Convert the operand using uuencode}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
861
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
862 \lineiv{zlib_codec}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
863 {zip, zlib}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
864 {byte string}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
865 {Compress the operand using gzip}
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
866
55141144dab5 Document standard encodings.
Martin v. Löwis <martin@v.loewis.de>
parents: 26294
diff changeset
867 \end{tableiv}