cpython-lru_cache-weakref: Lib/codecs.py annotate

annotate Lib/codecs.py @ 28646:193b8aec00ce legacy-trunk

Changes from Jonathan Riehl to allow his pgen extension (PEP 269) to work. This includes some more code that used to be part of pgen in the main parser; I'm okay with that. I'll see if the Windows build needs work next.

author	Guido van Rossum <guido@python.org>
date	Thu, 17 Apr 2003 14:55:42 +0000
parents	74d9604e9353
children	26a80f0af9f3

rev	line source
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	1 """ codecs -- Python Codec Registry, API and helpers.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	2
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	3
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	4 Written by Marc-Andre Lemburg (mal@lemburg.com).
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	5
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	6 (c) Copyright CNRI, All Rights Reserved. NO WARRANTY.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	7
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	8 """#"
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	9
26616 20e0755f3554 Patch to make _codecs a builtin module. This is necessary since Marc-André Lemburg <mal@egenix.com> parents: 26293 diff changeset	10 import __builtin__, sys
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	11
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	12 ### Registry and builtin stateless codec functions
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	13
12195 bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup Guido van Rossum <guido@python.org> parents: 12051 diff changeset	14 try:
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup Guido van Rossum <guido@python.org> parents: 12051 diff changeset	15 from _codecs import *
17839 c872fdf10130 Just changed "x,y" to "x, y" everywhere (i.e., inserted horizontal space Tim Peters <tim.peters@gmail.com> parents: 16070 diff changeset	16 except ImportError, why:
12195 bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup Guido van Rossum <guido@python.org> parents: 12051 diff changeset	17 raise SystemError,\
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup Guido van Rossum <guido@python.org> parents: 12051 diff changeset	18 'Failed to load the builtin codecs: %s' % why
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	19
17839 c872fdf10130 Just changed "x,y" to "x, y" everywhere (i.e., inserted horizontal space Tim Peters <tim.peters@gmail.com> parents: 16070 diff changeset	20 __all__ = ["register", "lookup", "open", "EncodedFile", "BOM", "BOM_BE",
23658 94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE, Walter Dörwald <walter@livinglogic.de> parents: 23620 diff changeset	21 "BOM_LE", "BOM32_BE", "BOM32_LE", "BOM64_BE", "BOM64_LE",
94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE, Walter Dörwald <walter@livinglogic.de> parents: 23620 diff changeset	22 "BOM_UTF8", "BOM_UTF16", "BOM_UTF16_LE", "BOM_UTF16_BE",
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 23658 diff changeset	23 "BOM_UTF32", "BOM_UTF32_LE", "BOM_UTF32_BE",
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 23658 diff changeset	24 "strict_errors", "ignore_errors", "replace_errors",
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 23658 diff changeset	25 "xmlcharrefreplace_errors",
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 23658 diff changeset	26 "register_error", "lookup_error"]
16070 b5345e99596b added __all__ lists to a number of Python modules Skip Montanaro <skip@pobox.com> parents: 15867 diff changeset	27
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	28 ### Constants
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	29
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	30 #
23658 94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE, Walter Dörwald <walter@livinglogic.de> parents: 23620 diff changeset	31 # Byte Order Mark (BOM = ZERO WIDTH NO-BREAK SPACE = U+FEFF)
94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE, Walter Dörwald <walter@livinglogic.de> parents: 23620 diff changeset	32 # and its possible byte string values
94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE, Walter Dörwald <walter@livinglogic.de> parents: 23620 diff changeset	33 # for UTF8/UTF16/UTF32 output and little/big endian machines
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	34 #
23658 94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE, Walter Dörwald <walter@livinglogic.de> parents: 23620 diff changeset	35
94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE, Walter Dörwald <walter@livinglogic.de> parents: 23620 diff changeset	36 # UTF-8
94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE, Walter Dörwald <walter@livinglogic.de> parents: 23620 diff changeset	37 BOM_UTF8 = '\xef\xbb\xbf'
94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE, Walter Dörwald <walter@livinglogic.de> parents: 23620 diff changeset	38
94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE, Walter Dörwald <walter@livinglogic.de> parents: 23620 diff changeset	39 # UTF-16, little endian
94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE, Walter Dörwald <walter@livinglogic.de> parents: 23620 diff changeset	40 BOM_LE = BOM_UTF16_LE = '\xff\xfe'
94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE, Walter Dörwald <walter@livinglogic.de> parents: 23620 diff changeset	41
94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE, Walter Dörwald <walter@livinglogic.de> parents: 23620 diff changeset	42 # UTF-16, big endian
94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE, Walter Dörwald <walter@livinglogic.de> parents: 23620 diff changeset	43 BOM_BE = BOM_UTF16_BE = '\xfe\xff'
94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE, Walter Dörwald <walter@livinglogic.de> parents: 23620 diff changeset	44
94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE, Walter Dörwald <walter@livinglogic.de> parents: 23620 diff changeset	45 # UTF-32, little endian
94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE, Walter Dörwald <walter@livinglogic.de> parents: 23620 diff changeset	46 BOM_UTF32_LE = '\xff\xfe\x00\x00'
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	47
23658 94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE, Walter Dörwald <walter@livinglogic.de> parents: 23620 diff changeset	48 # UTF-32, big endian
94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE, Walter Dörwald <walter@livinglogic.de> parents: 23620 diff changeset	49 BOM_UTF32_BE = '\x00\x00\xfe\xff'
94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE, Walter Dörwald <walter@livinglogic.de> parents: 23620 diff changeset	50
26616 20e0755f3554 Patch to make _codecs a builtin module. This is necessary since Marc-André Lemburg <mal@egenix.com> parents: 26293 diff changeset	51 if sys.byteorder == 'little':
20e0755f3554 Patch to make _codecs a builtin module. This is necessary since Marc-André Lemburg <mal@egenix.com> parents: 26293 diff changeset	52
20e0755f3554 Patch to make _codecs a builtin module. This is necessary since Marc-André Lemburg <mal@egenix.com> parents: 26293 diff changeset	53 # UTF-16, native endianness
20e0755f3554 Patch to make _codecs a builtin module. This is necessary since Marc-André Lemburg <mal@egenix.com> parents: 26293 diff changeset	54 BOM = BOM_UTF16 = BOM_UTF16_LE
20e0755f3554 Patch to make _codecs a builtin module. This is necessary since Marc-André Lemburg <mal@egenix.com> parents: 26293 diff changeset	55
20e0755f3554 Patch to make _codecs a builtin module. This is necessary since Marc-André Lemburg <mal@egenix.com> parents: 26293 diff changeset	56 # UTF-32, native endianness
20e0755f3554 Patch to make _codecs a builtin module. This is necessary since Marc-André Lemburg <mal@egenix.com> parents: 26293 diff changeset	57 BOM_UTF32 = BOM_UTF32_LE
23658 94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE, Walter Dörwald <walter@livinglogic.de> parents: 23620 diff changeset	58
26616 20e0755f3554 Patch to make _codecs a builtin module. This is necessary since Marc-André Lemburg <mal@egenix.com> parents: 26293 diff changeset	59 else:
20e0755f3554 Patch to make _codecs a builtin module. This is necessary since Marc-André Lemburg <mal@egenix.com> parents: 26293 diff changeset	60
20e0755f3554 Patch to make _codecs a builtin module. This is necessary since Marc-André Lemburg <mal@egenix.com> parents: 26293 diff changeset	61 # UTF-16, native endianness
20e0755f3554 Patch to make _codecs a builtin module. This is necessary since Marc-André Lemburg <mal@egenix.com> parents: 26293 diff changeset	62 BOM = BOM_UTF16 = BOM_UTF16_BE
20e0755f3554 Patch to make _codecs a builtin module. This is necessary since Marc-André Lemburg <mal@egenix.com> parents: 26293 diff changeset	63
20e0755f3554 Patch to make _codecs a builtin module. This is necessary since Marc-André Lemburg <mal@egenix.com> parents: 26293 diff changeset	64 # UTF-32, native endianness
20e0755f3554 Patch to make _codecs a builtin module. This is necessary since Marc-André Lemburg <mal@egenix.com> parents: 26293 diff changeset	65 BOM_UTF32 = BOM_UTF32_BE
23658 94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE, Walter Dörwald <walter@livinglogic.de> parents: 23620 diff changeset	66
94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE, Walter Dörwald <walter@livinglogic.de> parents: 23620 diff changeset	67 # Old broken names (don't use in new code)
94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE, Walter Dörwald <walter@livinglogic.de> parents: 23620 diff changeset	68 BOM32_LE = BOM_UTF16_LE
94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE, Walter Dörwald <walter@livinglogic.de> parents: 23620 diff changeset	69 BOM32_BE = BOM_UTF16_BE
94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE, Walter Dörwald <walter@livinglogic.de> parents: 23620 diff changeset	70 BOM64_LE = BOM_UTF32_LE
94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE, Walter Dörwald <walter@livinglogic.de> parents: 23620 diff changeset	71 BOM64_BE = BOM_UTF32_BE
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	72
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	73
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	74 ### Codec base classes (defining the API)
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	75
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	76 class Codec:
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	77
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	78 """ Defines the interface for stateless encoders/decoders.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	79
26293 ff5b510eba6b Add missing documentation for the PEP 293 functionality to Walter Dörwald <walter@livinglogic.de> parents: 26082 diff changeset	80 The .encode()/.decode() methods may use different error
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	81 handling schemes by providing the errors argument. These
26293 ff5b510eba6b Add missing documentation for the PEP 293 functionality to Walter Dörwald <walter@livinglogic.de> parents: 26082 diff changeset	82 string values are predefined:
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	83
12051 d89556c04ef9 Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12011 diff changeset	84 'strict' - raise a ValueError error (or a subclass)
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	85 'ignore' - ignore the character and continue with the next
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	86 'replace' - replace with a suitable replacement character;
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	87 Python will use the official U+FFFD REPLACEMENT
26293 ff5b510eba6b Add missing documentation for the PEP 293 functionality to Walter Dörwald <walter@livinglogic.de> parents: 26082 diff changeset	88 CHARACTER for the builtin Unicode codecs on
ff5b510eba6b Add missing documentation for the PEP 293 functionality to Walter Dörwald <walter@livinglogic.de> parents: 26082 diff changeset	89 decoding and '?' on encoding.
ff5b510eba6b Add missing documentation for the PEP 293 functionality to Walter Dörwald <walter@livinglogic.de> parents: 26082 diff changeset	90 'xmlcharrefreplace' - Replace with the appropriate XML
ff5b510eba6b Add missing documentation for the PEP 293 functionality to Walter Dörwald <walter@livinglogic.de> parents: 26082 diff changeset	91 character reference (only for encoding).
ff5b510eba6b Add missing documentation for the PEP 293 functionality to Walter Dörwald <walter@livinglogic.de> parents: 26082 diff changeset	92 'backslashreplace' - Replace with backslashed escape sequences
ff5b510eba6b Add missing documentation for the PEP 293 functionality to Walter Dörwald <walter@livinglogic.de> parents: 26082 diff changeset	93 (only for encoding).
ff5b510eba6b Add missing documentation for the PEP 293 functionality to Walter Dörwald <walter@livinglogic.de> parents: 26082 diff changeset	94
ff5b510eba6b Add missing documentation for the PEP 293 functionality to Walter Dörwald <walter@livinglogic.de> parents: 26082 diff changeset	95 The set of allowed values can be extended via register_error.
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	96
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	97 """
17839 c872fdf10130 Just changed "x,y" to "x, y" everywhere (i.e., inserted horizontal space Tim Peters <tim.peters@gmail.com> parents: 16070 diff changeset	98 def encode(self, input, errors='strict'):
12373 d63b57ac617d Deleted trailing whitespace. This is really a way to be able to add Guido van Rossum <guido@python.org> parents: 12367 diff changeset	99
12010 bd1014215f62 "intput" --> "input" (in docstring); noted by Detlef Lannert Fred Drake <fdrake@acm.org> parents: 11990 diff changeset	100 """ Encodes the object input and returns a tuple (output
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	101 object, length consumed).
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	102
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	103 errors defines the error handling to apply. It defaults to
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	104 'strict' handling.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	105
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	106 The method may not store state in the Codec instance. Use
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	107 StreamCodec for codecs which have to keep state in order to
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	108 make encoding/decoding efficient.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	109
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	110 The encoder must be able to handle zero length input and
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	111 return an empty object of the output object type in this
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	112 situation.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	113
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	114 """
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	115 raise NotImplementedError
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	116
17839 c872fdf10130 Just changed "x,y" to "x, y" everywhere (i.e., inserted horizontal space Tim Peters <tim.peters@gmail.com> parents: 16070 diff changeset	117 def decode(self, input, errors='strict'):
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	118
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	119 """ Decodes the object input and returns a tuple (output
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	120 object, length consumed).
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	121
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	122 input must be an object which provides the bf_getreadbuf
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	123 buffer slot. Python strings, buffer objects and memory
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	124 mapped files are examples of objects providing this slot.
12373 d63b57ac617d Deleted trailing whitespace. This is really a way to be able to add Guido van Rossum <guido@python.org> parents: 12367 diff changeset	125
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	126 errors defines the error handling to apply. It defaults to
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	127 'strict' handling.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	128
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	129 The method may not store state in the Codec instance. Use
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	130 StreamCodec for codecs which have to keep state in order to
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	131 make encoding/decoding efficient.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	132
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	133 The decoder must be able to handle zero length input and
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	134 return an empty object of the output object type in this
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	135 situation.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	136
12373 d63b57ac617d Deleted trailing whitespace. This is really a way to be able to add Guido van Rossum <guido@python.org> parents: 12367 diff changeset	137 """
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	138 raise NotImplementedError
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	139
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	140 #
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	141 # The StreamWriter and StreamReader class provide generic working
19967 0b51e607f333 Fix typo in comment Andrew M. Kuchling <amk@amk.ca> parents: 18936 diff changeset	142 # interfaces which can be used to implement new encoding submodules
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	143 # very easily. See encodings/utf_8.py for an example on how this is
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	144 # done.
12373 d63b57ac617d Deleted trailing whitespace. This is really a way to be able to add Guido van Rossum <guido@python.org> parents: 12367 diff changeset	145 #
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	146
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	147 class StreamWriter(Codec):
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	148
17839 c872fdf10130 Just changed "x,y" to "x, y" everywhere (i.e., inserted horizontal space Tim Peters <tim.peters@gmail.com> parents: 16070 diff changeset	149 def __init__(self, stream, errors='strict'):
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	150
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	151 """ Creates a StreamWriter instance.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	152
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	153 stream must be a file-like object open for writing
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	154 (binary) data.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	155
26293 ff5b510eba6b Add missing documentation for the PEP 293 functionality to Walter Dörwald <walter@livinglogic.de> parents: 26082 diff changeset	156 The StreamWriter may use different error handling
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	157 schemes by providing the errors keyword argument. These
26293 ff5b510eba6b Add missing documentation for the PEP 293 functionality to Walter Dörwald <walter@livinglogic.de> parents: 26082 diff changeset	158 parameters are predefined:
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	159
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	160 'strict' - raise a ValueError (or a subclass)
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	161 'ignore' - ignore the character and continue with the next
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	162 'replace'- replace with a suitable replacement character
26293 ff5b510eba6b Add missing documentation for the PEP 293 functionality to Walter Dörwald <walter@livinglogic.de> parents: 26082 diff changeset	163 'xmlcharrefreplace' - Replace with the appropriate XML
ff5b510eba6b Add missing documentation for the PEP 293 functionality to Walter Dörwald <walter@livinglogic.de> parents: 26082 diff changeset	164 character reference.
ff5b510eba6b Add missing documentation for the PEP 293 functionality to Walter Dörwald <walter@livinglogic.de> parents: 26082 diff changeset	165 'backslashreplace' - Replace with backslashed escape
ff5b510eba6b Add missing documentation for the PEP 293 functionality to Walter Dörwald <walter@livinglogic.de> parents: 26082 diff changeset	166 sequences (only for encoding).
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	167
26293 ff5b510eba6b Add missing documentation for the PEP 293 functionality to Walter Dörwald <walter@livinglogic.de> parents: 26082 diff changeset	168 The set of allowed parameter values can be extended via
ff5b510eba6b Add missing documentation for the PEP 293 functionality to Walter Dörwald <walter@livinglogic.de> parents: 26082 diff changeset	169 register_error.
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	170 """
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	171 self.stream = stream
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	172 self.errors = errors
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	173
12367 677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	174 def write(self, object):
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	175
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	176 """ Writes the object's contents encoded to self.stream.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	177 """
17839 c872fdf10130 Just changed "x,y" to "x, y" everywhere (i.e., inserted horizontal space Tim Peters <tim.peters@gmail.com> parents: 16070 diff changeset	178 data, consumed = self.encode(object, self.errors)
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	179 self.stream.write(data)
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	180
12367 677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	181 def writelines(self, list):
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	182
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	183 """ Writes the concatenated list of strings to the stream
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	184 using .write().
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	185 """
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	186 self.write(''.join(list))
12373 d63b57ac617d Deleted trailing whitespace. This is really a way to be able to add Guido van Rossum <guido@python.org> parents: 12367 diff changeset	187
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	188 def reset(self):
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	189
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	190 """ Flushes and resets the codec buffers used for keeping state.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	191
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	192 Calling this method should ensure that the data on the
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	193 output is put into a clean state, that allows appending
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	194 of new fresh data without having to rescan the whole
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	195 stream to recover state.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	196
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	197 """
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	198 pass
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	199
17839 c872fdf10130 Just changed "x,y" to "x, y" everywhere (i.e., inserted horizontal space Tim Peters <tim.peters@gmail.com> parents: 16070 diff changeset	200 def __getattr__(self, name,
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	201 getattr=getattr):
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	202
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	203 """ Inherit all other methods from the underlying stream.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	204 """
17839 c872fdf10130 Just changed "x,y" to "x, y" everywhere (i.e., inserted horizontal space Tim Peters <tim.peters@gmail.com> parents: 16070 diff changeset	205 return getattr(self.stream, name)
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	206
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	207 ###
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	208
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	209 class StreamReader(Codec):
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	210
17839 c872fdf10130 Just changed "x,y" to "x, y" everywhere (i.e., inserted horizontal space Tim Peters <tim.peters@gmail.com> parents: 16070 diff changeset	211 def __init__(self, stream, errors='strict'):
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	212
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	213 """ Creates a StreamReader instance.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	214
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	215 stream must be a file-like object open for reading
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	216 (binary) data.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	217
26293 ff5b510eba6b Add missing documentation for the PEP 293 functionality to Walter Dörwald <walter@livinglogic.de> parents: 26082 diff changeset	218 The StreamReader may use different error handling
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	219 schemes by providing the errors keyword argument. These
26293 ff5b510eba6b Add missing documentation for the PEP 293 functionality to Walter Dörwald <walter@livinglogic.de> parents: 26082 diff changeset	220 parameters are predefined:
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	221
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	222 'strict' - raise a ValueError (or a subclass)
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	223 'ignore' - ignore the character and continue with the next
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	224 'replace'- replace with a suitable replacement character;
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	225
26293 ff5b510eba6b Add missing documentation for the PEP 293 functionality to Walter Dörwald <walter@livinglogic.de> parents: 26082 diff changeset	226 The set of allowed parameter values can be extended via
ff5b510eba6b Add missing documentation for the PEP 293 functionality to Walter Dörwald <walter@livinglogic.de> parents: 26082 diff changeset	227 register_error.
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	228 """
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	229 self.stream = stream
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	230 self.errors = errors
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	231
12367 677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	232 def read(self, size=-1):
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	233
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	234 """ Decodes data from the stream self.stream and returns the
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	235 resulting object.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	236
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	237 size indicates the approximate maximum number of bytes to
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	238 read from the stream for decoding purposes. The decoder
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	239 can modify this setting as appropriate. The default value
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	240 -1 indicates to read and decode as much as possible. size
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	241 is intended to prevent having to decode huge files in one
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	242 step.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	243
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	244 The method should use a greedy read strategy meaning that
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	245 it should read as much data as is allowed within the
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	246 definition of the encoding and the given size, e.g. if
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	247 optional encoding endings or state markers are available
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	248 on the stream, these should be read too.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	249
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	250 """
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	251 # Unsliced reading:
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	252 if size < 0:
15498 46275611cd59 (Patch #102698) Fix for a bug reported by Wade Leftwich: Andrew M. Kuchling <amk@amk.ca> parents: 14978 diff changeset	253 return self.decode(self.stream.read(), self.errors)[0]
12373 d63b57ac617d Deleted trailing whitespace. This is really a way to be able to add Guido van Rossum <guido@python.org> parents: 12367 diff changeset	254
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	255 # Sliced reading:
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	256 read = self.stream.read
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	257 decode = self.decode
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	258 data = read(size)
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	259 i = 0
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	260 while 1:
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	261 try:
15498 46275611cd59 (Patch #102698) Fix for a bug reported by Wade Leftwich: Andrew M. Kuchling <amk@amk.ca> parents: 14978 diff changeset	262 object, decodedbytes = decode(data, self.errors)
17839 c872fdf10130 Just changed "x,y" to "x, y" everywhere (i.e., inserted horizontal space Tim Peters <tim.peters@gmail.com> parents: 16070 diff changeset	263 except ValueError, why:
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	264 # This method is slow but should work under pretty much
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	265 # all conditions; at most 10 tries are made
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	266 i = i + 1
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	267 newdata = read(1)
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	268 if not newdata or i > 10:
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	269 raise
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	270 data = data + newdata
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	271 else:
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	272 return object
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	273
12367 677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	274 def readline(self, size=None):
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	275
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	276 """ Read one line from the input stream and return the
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	277 decoded data.
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	278
12389 e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	279 Note: Unlike the .readlines() method, this method inherits
e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	280 the line breaking knowledge from the underlying stream's
e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	281 .readline() method -- there is currently no support for
e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	282 line breaking using the codec decoder due to lack of line
27618 74d9604e9353 Fix typos. Walter Dörwald <walter@livinglogic.de> parents: 26906 diff changeset	283 buffering. Subclasses should however, if possible, try to
12389 e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	284 implement this method using their own knowledge of line
e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	285 breaking.
12367 677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	286
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	287 size, if given, is passed as size argument to the stream's
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	288 .readline() method.
12373 d63b57ac617d Deleted trailing whitespace. This is really a way to be able to add Guido van Rossum <guido@python.org> parents: 12367 diff changeset	289
12367 677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	290 """
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	291 if size is None:
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	292 line = self.stream.readline()
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	293 else:
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	294 line = self.stream.readline(size)
17839 c872fdf10130 Just changed "x,y" to "x, y" everywhere (i.e., inserted horizontal space Tim Peters <tim.peters@gmail.com> parents: 16070 diff changeset	295 return self.decode(line, self.errors)[0]
12373 d63b57ac617d Deleted trailing whitespace. This is really a way to be able to add Guido van Rossum <guido@python.org> parents: 12367 diff changeset	296
12367 677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	297
22149 1e60b1e18145 backport loewis' checkin of Michael W. Hudson <mwh@python.net> parents: 19981 diff changeset	298 def readlines(self, sizehint=None):
12367 677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	299
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	300 """ Read all lines available on the input stream
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	301 and return them as list of lines.
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	302
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	303 Line breaks are implemented using the codec's decoder
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	304 method and are included in the list entries.
12373 d63b57ac617d Deleted trailing whitespace. This is really a way to be able to add Guido van Rossum <guido@python.org> parents: 12367 diff changeset	305
12367 677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	306 sizehint, if given, is passed as size argument to the
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	307 stream's .read() method.
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	308
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	309 """
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	310 if sizehint is None:
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	311 data = self.stream.read()
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	312 else:
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	313 data = self.stream.read(sizehint)
17839 c872fdf10130 Just changed "x,y" to "x, y" everywhere (i.e., inserted horizontal space Tim Peters <tim.peters@gmail.com> parents: 16070 diff changeset	314 return self.decode(data, self.errors)[0].splitlines(1)
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	315
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	316 def reset(self):
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	317
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	318 """ Resets the codec buffers used for keeping state.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	319
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	320 Note that no stream repositioning should take place.
13620 5b727ab37092 Spelling fixes supplied by Rob W. W. Hooft. All these are fixes in either Thomas Wouters <thomas@python.org> parents: 12986 diff changeset	321 This method is primarily intended to be able to recover
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	322 from decoding errors.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	323
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	324 """
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	325 pass
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	326
26082 f3a64035ba8b Add next() and __iter__() methods to StreamReader, StreamReaderWriter Walter Dörwald <walter@livinglogic.de> parents: 25091 diff changeset	327 def next(self):
f3a64035ba8b Add next() and __iter__() methods to StreamReader, StreamReaderWriter Walter Dörwald <walter@livinglogic.de> parents: 25091 diff changeset	328
f3a64035ba8b Add next() and __iter__() methods to StreamReader, StreamReaderWriter Walter Dörwald <walter@livinglogic.de> parents: 25091 diff changeset	329 """ Return the next decoded line from the input stream."""
f3a64035ba8b Add next() and __iter__() methods to StreamReader, StreamReaderWriter Walter Dörwald <walter@livinglogic.de> parents: 25091 diff changeset	330 line = self.readline()
f3a64035ba8b Add next() and __iter__() methods to StreamReader, StreamReaderWriter Walter Dörwald <walter@livinglogic.de> parents: 25091 diff changeset	331 if line:
f3a64035ba8b Add next() and __iter__() methods to StreamReader, StreamReaderWriter Walter Dörwald <walter@livinglogic.de> parents: 25091 diff changeset	332 return line
f3a64035ba8b Add next() and __iter__() methods to StreamReader, StreamReaderWriter Walter Dörwald <walter@livinglogic.de> parents: 25091 diff changeset	333 raise StopIteration
f3a64035ba8b Add next() and __iter__() methods to StreamReader, StreamReaderWriter Walter Dörwald <walter@livinglogic.de> parents: 25091 diff changeset	334
f3a64035ba8b Add next() and __iter__() methods to StreamReader, StreamReaderWriter Walter Dörwald <walter@livinglogic.de> parents: 25091 diff changeset	335 def __iter__(self):
f3a64035ba8b Add next() and __iter__() methods to StreamReader, StreamReaderWriter Walter Dörwald <walter@livinglogic.de> parents: 25091 diff changeset	336 return self
f3a64035ba8b Add next() and __iter__() methods to StreamReader, StreamReaderWriter Walter Dörwald <walter@livinglogic.de> parents: 25091 diff changeset	337
17839 c872fdf10130 Just changed "x,y" to "x, y" everywhere (i.e., inserted horizontal space Tim Peters <tim.peters@gmail.com> parents: 16070 diff changeset	338 def __getattr__(self, name,
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	339 getattr=getattr):
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	340
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	341 """ Inherit all other methods from the underlying stream.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	342 """
17839 c872fdf10130 Just changed "x,y" to "x, y" everywhere (i.e., inserted horizontal space Tim Peters <tim.peters@gmail.com> parents: 16070 diff changeset	343 return getattr(self.stream, name)
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	344
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	345 ###
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	346
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	347 class StreamReaderWriter:
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	348
12389 e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	349 """ StreamReaderWriter instances allow wrapping streams which
e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	350 work in both read and write modes.
e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	351
e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	352 The design is such that one can use the factory functions
13620 5b727ab37092 Spelling fixes supplied by Rob W. W. Hooft. All these are fixes in either Thomas Wouters <thomas@python.org> parents: 12986 diff changeset	353 returned by the codec.lookup() function to construct the
12389 e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	354 instance.
e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	355
e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	356 """
12367 677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	357 # Optional attributes set by the file wrappers below
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	358 encoding = 'unknown'
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	359
17839 c872fdf10130 Just changed "x,y" to "x, y" everywhere (i.e., inserted horizontal space Tim Peters <tim.peters@gmail.com> parents: 16070 diff changeset	360 def __init__(self, stream, Reader, Writer, errors='strict'):
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	361
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	362 """ Creates a StreamReaderWriter instance.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	363
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	364 stream must be a Stream-like object.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	365
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	366 Reader, Writer must be factory functions or classes
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	367 providing the StreamReader, StreamWriter interface resp.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	368
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	369 Error handling is done in the same way as defined for the
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	370 StreamWriter/Readers.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	371
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	372 """
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	373 self.stream = stream
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	374 self.reader = Reader(stream, errors)
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	375 self.writer = Writer(stream, errors)
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	376 self.errors = errors
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	377
17839 c872fdf10130 Just changed "x,y" to "x, y" everywhere (i.e., inserted horizontal space Tim Peters <tim.peters@gmail.com> parents: 16070 diff changeset	378 def read(self, size=-1):
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	379
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	380 return self.reader.read(size)
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	381
12534 189298a1fab4 Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12389 diff changeset	382 def readline(self, size=None):
12367 677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	383
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	384 return self.reader.readline(size)
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	385
12534 189298a1fab4 Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12389 diff changeset	386 def readlines(self, sizehint=None):
12367 677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	387
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	388 return self.reader.readlines(sizehint)
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	389
26082 f3a64035ba8b Add next() and __iter__() methods to StreamReader, StreamReaderWriter Walter Dörwald <walter@livinglogic.de> parents: 25091 diff changeset	390 def next(self):
f3a64035ba8b Add next() and __iter__() methods to StreamReader, StreamReaderWriter Walter Dörwald <walter@livinglogic.de> parents: 25091 diff changeset	391
f3a64035ba8b Add next() and __iter__() methods to StreamReader, StreamReaderWriter Walter Dörwald <walter@livinglogic.de> parents: 25091 diff changeset	392 """ Return the next decoded line from the input stream."""
f3a64035ba8b Add next() and __iter__() methods to StreamReader, StreamReaderWriter Walter Dörwald <walter@livinglogic.de> parents: 25091 diff changeset	393 return self.reader.next()
f3a64035ba8b Add next() and __iter__() methods to StreamReader, StreamReaderWriter Walter Dörwald <walter@livinglogic.de> parents: 25091 diff changeset	394
f3a64035ba8b Add next() and __iter__() methods to StreamReader, StreamReaderWriter Walter Dörwald <walter@livinglogic.de> parents: 25091 diff changeset	395 def __iter__(self):
f3a64035ba8b Add next() and __iter__() methods to StreamReader, StreamReaderWriter Walter Dörwald <walter@livinglogic.de> parents: 25091 diff changeset	396 return self
f3a64035ba8b Add next() and __iter__() methods to StreamReader, StreamReaderWriter Walter Dörwald <walter@livinglogic.de> parents: 25091 diff changeset	397
17839 c872fdf10130 Just changed "x,y" to "x, y" everywhere (i.e., inserted horizontal space Tim Peters <tim.peters@gmail.com> parents: 16070 diff changeset	398 def write(self, data):
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	399
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	400 return self.writer.write(data)
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	401
17839 c872fdf10130 Just changed "x,y" to "x, y" everywhere (i.e., inserted horizontal space Tim Peters <tim.peters@gmail.com> parents: 16070 diff changeset	402 def writelines(self, list):
12367 677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	403
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	404 return self.writer.writelines(list)
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	405
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	406 def reset(self):
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	407
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	408 self.reader.reset()
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	409 self.writer.reset()
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	410
17839 c872fdf10130 Just changed "x,y" to "x, y" everywhere (i.e., inserted horizontal space Tim Peters <tim.peters@gmail.com> parents: 16070 diff changeset	411 def __getattr__(self, name,
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	412 getattr=getattr):
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	413
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	414 """ Inherit all other methods from the underlying stream.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	415 """
17839 c872fdf10130 Just changed "x,y" to "x, y" everywhere (i.e., inserted horizontal space Tim Peters <tim.peters@gmail.com> parents: 16070 diff changeset	416 return getattr(self.stream, name)
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	417
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	418 ###
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	419
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	420 class StreamRecoder:
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	421
12389 e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	422 """ StreamRecoder instances provide a frontend - backend
e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	423 view of encoding data.
e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	424
e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	425 They use the complete set of APIs returned by the
e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	426 codecs.lookup() function to implement their task.
e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	427
e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	428 Data written to the stream is first decoded into an
e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	429 intermediate format (which is dependent on the given codec
e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	430 combination) and then written to the stream using an instance
e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	431 of the provided Writer class.
e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	432
e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	433 In the other direction, data is read from the stream using a
e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	434 Reader instance and then return encoded data to the caller.
e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	435
e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	436 """
12367 677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	437 # Optional attributes set by the file wrappers below
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	438 data_encoding = 'unknown'
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	439 file_encoding = 'unknown'
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	440
17839 c872fdf10130 Just changed "x,y" to "x, y" everywhere (i.e., inserted horizontal space Tim Peters <tim.peters@gmail.com> parents: 16070 diff changeset	441 def __init__(self, stream, encode, decode, Reader, Writer,
c872fdf10130 Just changed "x,y" to "x, y" everywhere (i.e., inserted horizontal space Tim Peters <tim.peters@gmail.com> parents: 16070 diff changeset	442 errors='strict'):
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	443
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	444 """ Creates a StreamRecoder instance which implements a two-way
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	445 conversion: encode and decode work on the frontend (the
12373 d63b57ac617d Deleted trailing whitespace. This is really a way to be able to add Guido van Rossum <guido@python.org> parents: 12367 diff changeset	446 input to .read() and output of .write()) while
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	447 Reader and Writer work on the backend (reading and
12011 4fa25a292273 Oops, another in the same file; I should read the mail fully before Fred Drake <fdrake@acm.org> parents: 12010 diff changeset	448 writing to the stream).
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	449
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	450 You can use these objects to do transparent direct
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	451 recodings from e.g. latin-1 to utf-8 and back.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	452
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	453 stream must be a file-like object.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	454
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	455 encode, decode must adhere to the Codec interface, Reader,
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	456 Writer must be factory functions or classes providing the
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	457 StreamReader, StreamWriter interface resp.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	458
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	459 encode and decode are needed for the frontend translation,
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	460 Reader and Writer for the backend translation. Unicode is
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	461 used as intermediate encoding.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	462
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	463 Error handling is done in the same way as defined for the
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	464 StreamWriter/Readers.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	465
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	466 """
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	467 self.stream = stream
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	468 self.encode = encode
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	469 self.decode = decode
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	470 self.reader = Reader(stream, errors)
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	471 self.writer = Writer(stream, errors)
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	472 self.errors = errors
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	473
17839 c872fdf10130 Just changed "x,y" to "x, y" everywhere (i.e., inserted horizontal space Tim Peters <tim.peters@gmail.com> parents: 16070 diff changeset	474 def read(self, size=-1):
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	475
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	476 data = self.reader.read(size)
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	477 data, bytesencoded = self.encode(data, self.errors)
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	478 return data
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	479
17839 c872fdf10130 Just changed "x,y" to "x, y" everywhere (i.e., inserted horizontal space Tim Peters <tim.peters@gmail.com> parents: 16070 diff changeset	480 def readline(self, size=None):
12367 677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	481
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	482 if size is None:
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	483 data = self.reader.readline()
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	484 else:
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	485 data = self.reader.readline(size)
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	486 data, bytesencoded = self.encode(data, self.errors)
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	487 return data
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	488
17839 c872fdf10130 Just changed "x,y" to "x, y" everywhere (i.e., inserted horizontal space Tim Peters <tim.peters@gmail.com> parents: 16070 diff changeset	489 def readlines(self, sizehint=None):
12367 677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	490
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	491 if sizehint is None:
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	492 data = self.reader.read()
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	493 else:
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	494 data = self.reader.read(sizehint)
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	495 data, bytesencoded = self.encode(data, self.errors)
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	496 return data.splitlines(1)
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	497
26082 f3a64035ba8b Add next() and __iter__() methods to StreamReader, StreamReaderWriter Walter Dörwald <walter@livinglogic.de> parents: 25091 diff changeset	498 def next(self):
f3a64035ba8b Add next() and __iter__() methods to StreamReader, StreamReaderWriter Walter Dörwald <walter@livinglogic.de> parents: 25091 diff changeset	499
f3a64035ba8b Add next() and __iter__() methods to StreamReader, StreamReaderWriter Walter Dörwald <walter@livinglogic.de> parents: 25091 diff changeset	500 """ Return the next decoded line from the input stream."""
f3a64035ba8b Add next() and __iter__() methods to StreamReader, StreamReaderWriter Walter Dörwald <walter@livinglogic.de> parents: 25091 diff changeset	501 return self.reader.next()
f3a64035ba8b Add next() and __iter__() methods to StreamReader, StreamReaderWriter Walter Dörwald <walter@livinglogic.de> parents: 25091 diff changeset	502
f3a64035ba8b Add next() and __iter__() methods to StreamReader, StreamReaderWriter Walter Dörwald <walter@livinglogic.de> parents: 25091 diff changeset	503 def __iter__(self):
f3a64035ba8b Add next() and __iter__() methods to StreamReader, StreamReaderWriter Walter Dörwald <walter@livinglogic.de> parents: 25091 diff changeset	504 return self
f3a64035ba8b Add next() and __iter__() methods to StreamReader, StreamReaderWriter Walter Dörwald <walter@livinglogic.de> parents: 25091 diff changeset	505
17839 c872fdf10130 Just changed "x,y" to "x, y" everywhere (i.e., inserted horizontal space Tim Peters <tim.peters@gmail.com> parents: 16070 diff changeset	506 def write(self, data):
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	507
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	508 data, bytesdecoded = self.decode(data, self.errors)
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	509 return self.writer.write(data)
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	510
17839 c872fdf10130 Just changed "x,y" to "x, y" everywhere (i.e., inserted horizontal space Tim Peters <tim.peters@gmail.com> parents: 16070 diff changeset	511 def writelines(self, list):
12367 677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	512
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	513 data = ''.join(list)
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	514 data, bytesdecoded = self.decode(data, self.errors)
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	515 return self.writer.write(data)
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	516
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	517 def reset(self):
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	518
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	519 self.reader.reset()
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	520 self.writer.reset()
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	521
17839 c872fdf10130 Just changed "x,y" to "x, y" everywhere (i.e., inserted horizontal space Tim Peters <tim.peters@gmail.com> parents: 16070 diff changeset	522 def __getattr__(self, name,
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	523 getattr=getattr):
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	524
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	525 """ Inherit all other methods from the underlying stream.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	526 """
17839 c872fdf10130 Just changed "x,y" to "x, y" everywhere (i.e., inserted horizontal space Tim Peters <tim.peters@gmail.com> parents: 16070 diff changeset	527 return getattr(self.stream, name)
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	528
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	529 ### Shortcuts
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	530
12986 5279a6e00d36 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12534 diff changeset	531 def open(filename, mode='rb', encoding=None, errors='strict', buffering=1):
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	532
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	533 """ Open an encoded file using the given mode and return
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	534 a wrapped version providing transparent encoding/decoding.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	535
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	536 Note: The wrapped version will only accept the object format
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	537 defined by the codecs, i.e. Unicode objects for most builtin
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	538 codecs. Output is also codec dependent and will usually by
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	539 Unicode as well.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	540
12986 5279a6e00d36 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12534 diff changeset	541 Files are always opened in binary mode, even if no binary mode
27618 74d9604e9353 Fix typos. Walter Dörwald <walter@livinglogic.de> parents: 26906 diff changeset	542 was specified. This is done to avoid data loss due to encodings
12986 5279a6e00d36 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12534 diff changeset	543 using 8-bit values. The default file mode is 'rb' meaning to
5279a6e00d36 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12534 diff changeset	544 open the file in binary read mode.
5279a6e00d36 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12534 diff changeset	545
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	546 encoding specifies the encoding which is to be used for the
27618 74d9604e9353 Fix typos. Walter Dörwald <walter@livinglogic.de> parents: 26906 diff changeset	547 file.
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	548
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	549 errors may be given to define the error handling. It defaults
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	550 to 'strict' which causes ValueErrors to be raised in case an
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	551 encoding error occurs.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	552
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	553 buffering has the same meaning as for the builtin open() API.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	554 It defaults to line buffered.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	555
12389 e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	556 The returned wrapped file object provides an extra attribute
e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	557 .encoding which allows querying the used encoding. This
e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	558 attribute is only available if an encoding was specified as
e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	559 parameter.
e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	560
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	561 """
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	562 if encoding is not None and \
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	563 'b' not in mode:
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	564 # Force opening of the file in binary mode
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	565 mode = mode + 'b'
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	566 file = __builtin__.open(filename, mode, buffering)
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	567 if encoding is None:
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	568 return file
17839 c872fdf10130 Just changed "x,y" to "x, y" everywhere (i.e., inserted horizontal space Tim Peters <tim.peters@gmail.com> parents: 16070 diff changeset	569 (e, d, sr, sw) = lookup(encoding)
12367 677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	570 srw = StreamReaderWriter(file, sr, sw, errors)
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	571 # Add attributes to simplify introspection
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	572 srw.encoding = encoding
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	573 return srw
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	574
12367 677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	575 def EncodedFile(file, data_encoding, file_encoding=None, errors='strict'):
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	576
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	577 """ Return a wrapped version of file which provides transparent
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	578 encoding translation.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	579
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	580 Strings written to the wrapped file are interpreted according
12367 677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	581 to the given data_encoding and then written to the original
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	582 file as string using file_encoding. The intermediate encoding
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	583 will usually be Unicode but depends on the specified codecs.
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	584
12367 677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	585 Strings are read from the file using file_encoding and then
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	586 passed back to the caller as string using data_encoding.
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	587
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	588 If file_encoding is not given, it defaults to data_encoding.
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	589
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	590 errors may be given to define the error handling. It defaults
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	591 to 'strict' which causes ValueErrors to be raised in case an
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	592 encoding error occurs.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	593
12389 e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	594 The returned wrapped file object provides two extra attributes
e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	595 .data_encoding and .file_encoding which reflect the given
e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	596 parameters of the same name. The attributes can be used for
e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	597 introspection by Python programs.
e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	598
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	599 """
12367 677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	600 if file_encoding is None:
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	601 file_encoding = data_encoding
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	602 encode, decode = lookup(data_encoding)[:2]
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	603 Reader, Writer = lookup(file_encoding)[2:]
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	604 sr = StreamRecoder(file,
17839 c872fdf10130 Just changed "x,y" to "x, y" everywhere (i.e., inserted horizontal space Tim Peters <tim.peters@gmail.com> parents: 16070 diff changeset	605 encode, decode, Reader, Writer,
12367 677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	606 errors)
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	607 # Add attributes to simplify introspection
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	608 sr.data_encoding = data_encoding
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	609 sr.file_encoding = file_encoding
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	610 return sr
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	611
19981 0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	612 ### Helpers for codec lookup
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	613
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	614 def getencoder(encoding):
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	615
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	616 """ Lookup up the codec for the given encoding and return
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	617 its encoder function.
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	618
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	619 Raises a LookupError in case the encoding cannot be found.
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	620
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	621 """
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	622 return lookup(encoding)[0]
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	623
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	624 def getdecoder(encoding):
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	625
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	626 """ Lookup up the codec for the given encoding and return
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	627 its decoder function.
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	628
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	629 Raises a LookupError in case the encoding cannot be found.
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	630
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	631 """
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	632 return lookup(encoding)[1]
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	633
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	634 def getreader(encoding):
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	635
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	636 """ Lookup up the codec for the given encoding and return
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	637 its StreamReader class or factory function.
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	638
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	639 Raises a LookupError in case the encoding cannot be found.
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	640
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	641 """
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	642 return lookup(encoding)[2]
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	643
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	644 def getwriter(encoding):
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	645
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	646 """ Lookup up the codec for the given encoding and return
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	647 its StreamWriter class or factory function.
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	648
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	649 Raises a LookupError in case the encoding cannot be found.
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	650
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	651 """
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	652 return lookup(encoding)[3]
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	653
15706 bf9bb0e3dd34 This patch changes the default behaviour of the builtin charmap Marc-André Lemburg <mal@egenix.com> parents: 15498 diff changeset	654 ### Helpers for charmap-based codecs
bf9bb0e3dd34 This patch changes the default behaviour of the builtin charmap Marc-André Lemburg <mal@egenix.com> parents: 15498 diff changeset	655
bf9bb0e3dd34 This patch changes the default behaviour of the builtin charmap Marc-André Lemburg <mal@egenix.com> parents: 15498 diff changeset	656 def make_identity_dict(rng):
bf9bb0e3dd34 This patch changes the default behaviour of the builtin charmap Marc-André Lemburg <mal@egenix.com> parents: 15498 diff changeset	657
bf9bb0e3dd34 This patch changes the default behaviour of the builtin charmap Marc-André Lemburg <mal@egenix.com> parents: 15498 diff changeset	658 """ make_identity_dict(rng) -> dict
bf9bb0e3dd34 This patch changes the default behaviour of the builtin charmap Marc-André Lemburg <mal@egenix.com> parents: 15498 diff changeset	659
bf9bb0e3dd34 This patch changes the default behaviour of the builtin charmap Marc-André Lemburg <mal@egenix.com> parents: 15498 diff changeset	660 Return a dictionary where elements of the rng sequence are
bf9bb0e3dd34 This patch changes the default behaviour of the builtin charmap Marc-André Lemburg <mal@egenix.com> parents: 15498 diff changeset	661 mapped to themselves.
15867 f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15706 diff changeset	662
15706 bf9bb0e3dd34 This patch changes the default behaviour of the builtin charmap Marc-André Lemburg <mal@egenix.com> parents: 15498 diff changeset	663 """
bf9bb0e3dd34 This patch changes the default behaviour of the builtin charmap Marc-André Lemburg <mal@egenix.com> parents: 15498 diff changeset	664 res = {}
bf9bb0e3dd34 This patch changes the default behaviour of the builtin charmap Marc-André Lemburg <mal@egenix.com> parents: 15498 diff changeset	665 for i in rng:
bf9bb0e3dd34 This patch changes the default behaviour of the builtin charmap Marc-André Lemburg <mal@egenix.com> parents: 15498 diff changeset	666 res[i]=i
bf9bb0e3dd34 This patch changes the default behaviour of the builtin charmap Marc-André Lemburg <mal@egenix.com> parents: 15498 diff changeset	667 return res
bf9bb0e3dd34 This patch changes the default behaviour of the builtin charmap Marc-André Lemburg <mal@egenix.com> parents: 15498 diff changeset	668
17843 3e256f9d99ad Moved the encoding map building logic from the individual mapping Marc-André Lemburg <mal@egenix.com> parents: 17839 diff changeset	669 def make_encoding_map(decoding_map):
3e256f9d99ad Moved the encoding map building logic from the individual mapping Marc-André Lemburg <mal@egenix.com> parents: 17839 diff changeset	670
3e256f9d99ad Moved the encoding map building logic from the individual mapping Marc-André Lemburg <mal@egenix.com> parents: 17839 diff changeset	671 """ Creates an encoding map from a decoding map.
3e256f9d99ad Moved the encoding map building logic from the individual mapping Marc-André Lemburg <mal@egenix.com> parents: 17839 diff changeset	672
27618 74d9604e9353 Fix typos. Walter Dörwald <walter@livinglogic.de> parents: 26906 diff changeset	673 If a target mapping in the decoding map occurs multiple
17843 3e256f9d99ad Moved the encoding map building logic from the individual mapping Marc-André Lemburg <mal@egenix.com> parents: 17839 diff changeset	674 times, then that target is mapped to None (undefined mapping),
3e256f9d99ad Moved the encoding map building logic from the individual mapping Marc-André Lemburg <mal@egenix.com> parents: 17839 diff changeset	675 causing an exception when encountered by the charmap codec
3e256f9d99ad Moved the encoding map building logic from the individual mapping Marc-André Lemburg <mal@egenix.com> parents: 17839 diff changeset	676 during translation.
3e256f9d99ad Moved the encoding map building logic from the individual mapping Marc-André Lemburg <mal@egenix.com> parents: 17839 diff changeset	677
3e256f9d99ad Moved the encoding map building logic from the individual mapping Marc-André Lemburg <mal@egenix.com> parents: 17839 diff changeset	678 One example where this happens is cp875.py which decodes
3e256f9d99ad Moved the encoding map building logic from the individual mapping Marc-André Lemburg <mal@egenix.com> parents: 17839 diff changeset	679 multiple character to \u001a.
3e256f9d99ad Moved the encoding map building logic from the individual mapping Marc-André Lemburg <mal@egenix.com> parents: 17839 diff changeset	680
3e256f9d99ad Moved the encoding map building logic from the individual mapping Marc-André Lemburg <mal@egenix.com> parents: 17839 diff changeset	681 """
3e256f9d99ad Moved the encoding map building logic from the individual mapping Marc-André Lemburg <mal@egenix.com> parents: 17839 diff changeset	682 m = {}
3e256f9d99ad Moved the encoding map building logic from the individual mapping Marc-André Lemburg <mal@egenix.com> parents: 17839 diff changeset	683 for k,v in decoding_map.items():
23620 a1de0bab5475 SF 563203. Replaced 'has_key()' with 'in'. Raymond Hettinger <python@rcn.com> parents: 22149 diff changeset	684 if not v in m:
17843 3e256f9d99ad Moved the encoding map building logic from the individual mapping Marc-André Lemburg <mal@egenix.com> parents: 17839 diff changeset	685 m[v] = k
3e256f9d99ad Moved the encoding map building logic from the individual mapping Marc-André Lemburg <mal@egenix.com> parents: 17839 diff changeset	686 else:
3e256f9d99ad Moved the encoding map building logic from the individual mapping Marc-André Lemburg <mal@egenix.com> parents: 17839 diff changeset	687 m[v] = None
3e256f9d99ad Moved the encoding map building logic from the individual mapping Marc-André Lemburg <mal@egenix.com> parents: 17839 diff changeset	688 return m
17978 5e87a08abb05 Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 17843 diff changeset	689
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 23658 diff changeset	690 ### error handlers
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 23658 diff changeset	691
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 23658 diff changeset	692 strict_errors = lookup_error("strict")
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 23658 diff changeset	693 ignore_errors = lookup_error("ignore")
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 23658 diff changeset	694 replace_errors = lookup_error("replace")
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 23658 diff changeset	695 xmlcharrefreplace_errors = lookup_error("xmlcharrefreplace")
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 23658 diff changeset	696 backslashreplace_errors = lookup_error("backslashreplace")
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 23658 diff changeset	697
18904 5af1ee422575 Add dead imports of modules that are "magically" imported. Martin v. Löwis <martin@v.loewis.de> parents: 17978 diff changeset	698 # Tell modulefinder that using codecs probably needs the encodings
5af1ee422575 Add dead imports of modules that are "magically" imported. Martin v. Löwis <martin@v.loewis.de> parents: 17978 diff changeset	699 # package
5af1ee422575 Add dead imports of modules that are "magically" imported. Martin v. Löwis <martin@v.loewis.de> parents: 17978 diff changeset	700 _false = 0
5af1ee422575 Add dead imports of modules that are "magically" imported. Martin v. Löwis <martin@v.loewis.de> parents: 17978 diff changeset	701 if _false:
5af1ee422575 Add dead imports of modules that are "magically" imported. Martin v. Löwis <martin@v.loewis.de> parents: 17978 diff changeset	702 import encodings
5af1ee422575 Add dead imports of modules that are "magically" imported. Martin v. Löwis <martin@v.loewis.de> parents: 17978 diff changeset	703
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	704 ### Tests
12373 d63b57ac617d Deleted trailing whitespace. This is really a way to be able to add Guido van Rossum <guido@python.org> parents: 12367 diff changeset	705
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	706 if __name__ == '__main__':
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	707
12367 677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	708 # Make stdout translate Latin-1 output into UTF-8 output
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	709 sys.stdout = EncodedFile(sys.stdout, 'latin-1', 'utf-8')
12373 d63b57ac617d Deleted trailing whitespace. This is really a way to be able to add Guido van Rossum <guido@python.org> parents: 12367 diff changeset	710
12367 677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	711 # Have stdin translate Latin-1 input into UTF-8 input
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	712 sys.stdin = EncodedFile(sys.stdin, 'utf-8', 'latin-1')

Mercurial > cpython-lru_cache-weakref

annotate Lib/codecs.py @ 28646:193b8aec00ce legacy-trunk