cpython-lru_cache-weakref: Lib/codecs.py annotate

annotate Lib/codecs.py @ 58765:2ba7677bfa95 legacy-trunk

testCopysign was defined twice in test_math; combine the definitions

author	Mark Dickinson <dickinsm@gmail.com>
date	Sat, 06 Feb 2010 23:11:25 +0000
parents	5341b30b1812
children	7e3c47354da3

rev	line source
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	1 """ codecs -- Python Codec Registry, API and helpers.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	2
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	3
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	4 Written by Marc-Andre Lemburg (mal@lemburg.com).
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	5
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	6 (c) Copyright CNRI, All Rights Reserved. NO WARRANTY.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	7
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	8 """#"
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	9
26616 20e0755f3554 Patch to make _codecs a builtin module. This is necessary since Marc-André Lemburg <mal@egenix.com> parents: 26293 diff changeset	10 import __builtin__, sys
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	11
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	12 ### Registry and builtin stateless codec functions
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	13
12195 bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup Guido van Rossum <guido@python.org> parents: 12051 diff changeset	14 try:
bc861add5d71 Marc-Andre Lemburg: Error reporting in the codec registry and lookup Guido van Rossum <guido@python.org> parents: 12051 diff changeset	15 from _codecs import *
17839 c872fdf10130 Just changed "x,y" to "x, y" everywhere (i.e., inserted horizontal space Tim Peters <tim.peters@gmail.com> parents: 16070 diff changeset	16 except ImportError, why:
37080 6771f0e08b60 Change raise statement to PEP 8 style. Walter Dörwald <walter@livinglogic.de> parents: 37024 diff changeset	17 raise SystemError('Failed to load the builtin codecs: %s' % why)
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	18
17839 c872fdf10130 Just changed "x,y" to "x, y" everywhere (i.e., inserted horizontal space Tim Peters <tim.peters@gmail.com> parents: 16070 diff changeset	19 __all__ = ["register", "lookup", "open", "EncodedFile", "BOM", "BOM_BE",
23658 94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE, Walter Dörwald <walter@livinglogic.de> parents: 23620 diff changeset	20 "BOM_LE", "BOM32_BE", "BOM32_LE", "BOM64_BE", "BOM64_LE",
94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE, Walter Dörwald <walter@livinglogic.de> parents: 23620 diff changeset	21 "BOM_UTF8", "BOM_UTF16", "BOM_UTF16_LE", "BOM_UTF16_BE",
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 23658 diff changeset	22 "BOM_UTF32", "BOM_UTF32_LE", "BOM_UTF32_BE",
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 23658 diff changeset	23 "strict_errors", "ignore_errors", "replace_errors",
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 23658 diff changeset	24 "xmlcharrefreplace_errors",
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 23658 diff changeset	25 "register_error", "lookup_error"]
16070 b5345e99596b added __all__ lists to a number of Python modules Skip Montanaro <skip@pobox.com> parents: 15867 diff changeset	26
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	27 ### Constants
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	28
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	29 #
23658 94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE, Walter Dörwald <walter@livinglogic.de> parents: 23620 diff changeset	30 # Byte Order Mark (BOM = ZERO WIDTH NO-BREAK SPACE = U+FEFF)
94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE, Walter Dörwald <walter@livinglogic.de> parents: 23620 diff changeset	31 # and its possible byte string values
94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE, Walter Dörwald <walter@livinglogic.de> parents: 23620 diff changeset	32 # for UTF8/UTF16/UTF32 output and little/big endian machines
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	33 #
23658 94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE, Walter Dörwald <walter@livinglogic.de> parents: 23620 diff changeset	34
94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE, Walter Dörwald <walter@livinglogic.de> parents: 23620 diff changeset	35 # UTF-8
94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE, Walter Dörwald <walter@livinglogic.de> parents: 23620 diff changeset	36 BOM_UTF8 = '\xef\xbb\xbf'
94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE, Walter Dörwald <walter@livinglogic.de> parents: 23620 diff changeset	37
94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE, Walter Dörwald <walter@livinglogic.de> parents: 23620 diff changeset	38 # UTF-16, little endian
94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE, Walter Dörwald <walter@livinglogic.de> parents: 23620 diff changeset	39 BOM_LE = BOM_UTF16_LE = '\xff\xfe'
94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE, Walter Dörwald <walter@livinglogic.de> parents: 23620 diff changeset	40
94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE, Walter Dörwald <walter@livinglogic.de> parents: 23620 diff changeset	41 # UTF-16, big endian
94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE, Walter Dörwald <walter@livinglogic.de> parents: 23620 diff changeset	42 BOM_BE = BOM_UTF16_BE = '\xfe\xff'
94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE, Walter Dörwald <walter@livinglogic.de> parents: 23620 diff changeset	43
94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE, Walter Dörwald <walter@livinglogic.de> parents: 23620 diff changeset	44 # UTF-32, little endian
94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE, Walter Dörwald <walter@livinglogic.de> parents: 23620 diff changeset	45 BOM_UTF32_LE = '\xff\xfe\x00\x00'
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	46
23658 94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE, Walter Dörwald <walter@livinglogic.de> parents: 23620 diff changeset	47 # UTF-32, big endian
94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE, Walter Dörwald <walter@livinglogic.de> parents: 23620 diff changeset	48 BOM_UTF32_BE = '\x00\x00\xfe\xff'
94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE, Walter Dörwald <walter@livinglogic.de> parents: 23620 diff changeset	49
26616 20e0755f3554 Patch to make _codecs a builtin module. This is necessary since Marc-André Lemburg <mal@egenix.com> parents: 26293 diff changeset	50 if sys.byteorder == 'little':
20e0755f3554 Patch to make _codecs a builtin module. This is necessary since Marc-André Lemburg <mal@egenix.com> parents: 26293 diff changeset	51
20e0755f3554 Patch to make _codecs a builtin module. This is necessary since Marc-André Lemburg <mal@egenix.com> parents: 26293 diff changeset	52 # UTF-16, native endianness
20e0755f3554 Patch to make _codecs a builtin module. This is necessary since Marc-André Lemburg <mal@egenix.com> parents: 26293 diff changeset	53 BOM = BOM_UTF16 = BOM_UTF16_LE
20e0755f3554 Patch to make _codecs a builtin module. This is necessary since Marc-André Lemburg <mal@egenix.com> parents: 26293 diff changeset	54
20e0755f3554 Patch to make _codecs a builtin module. This is necessary since Marc-André Lemburg <mal@egenix.com> parents: 26293 diff changeset	55 # UTF-32, native endianness
20e0755f3554 Patch to make _codecs a builtin module. This is necessary since Marc-André Lemburg <mal@egenix.com> parents: 26293 diff changeset	56 BOM_UTF32 = BOM_UTF32_LE
23658 94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE, Walter Dörwald <walter@livinglogic.de> parents: 23620 diff changeset	57
26616 20e0755f3554 Patch to make _codecs a builtin module. This is necessary since Marc-André Lemburg <mal@egenix.com> parents: 26293 diff changeset	58 else:
20e0755f3554 Patch to make _codecs a builtin module. This is necessary since Marc-André Lemburg <mal@egenix.com> parents: 26293 diff changeset	59
20e0755f3554 Patch to make _codecs a builtin module. This is necessary since Marc-André Lemburg <mal@egenix.com> parents: 26293 diff changeset	60 # UTF-16, native endianness
20e0755f3554 Patch to make _codecs a builtin module. This is necessary since Marc-André Lemburg <mal@egenix.com> parents: 26293 diff changeset	61 BOM = BOM_UTF16 = BOM_UTF16_BE
20e0755f3554 Patch to make _codecs a builtin module. This is necessary since Marc-André Lemburg <mal@egenix.com> parents: 26293 diff changeset	62
20e0755f3554 Patch to make _codecs a builtin module. This is necessary since Marc-André Lemburg <mal@egenix.com> parents: 26293 diff changeset	63 # UTF-32, native endianness
20e0755f3554 Patch to make _codecs a builtin module. This is necessary since Marc-André Lemburg <mal@egenix.com> parents: 26293 diff changeset	64 BOM_UTF32 = BOM_UTF32_BE
23658 94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE, Walter Dörwald <walter@livinglogic.de> parents: 23620 diff changeset	65
94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE, Walter Dörwald <walter@livinglogic.de> parents: 23620 diff changeset	66 # Old broken names (don't use in new code)
94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE, Walter Dörwald <walter@livinglogic.de> parents: 23620 diff changeset	67 BOM32_LE = BOM_UTF16_LE
94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE, Walter Dörwald <walter@livinglogic.de> parents: 23620 diff changeset	68 BOM32_BE = BOM_UTF16_BE
94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE, Walter Dörwald <walter@livinglogic.de> parents: 23620 diff changeset	69 BOM64_LE = BOM_UTF32_LE
94aa4eb9bb7d Add constants BOM_UTF8, BOM_UTF16, BOM_UTF16_LE, BOM_UTF16_BE, Walter Dörwald <walter@livinglogic.de> parents: 23620 diff changeset	70 BOM64_BE = BOM_UTF32_BE
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	71
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	72
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	73 ### Codec base classes (defining the API)
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	74
36996 7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	75 class CodecInfo(tuple):
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	76
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	77 def __new__(cls, encode, decode, streamreader=None, streamwriter=None,
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	78 incrementalencoder=None, incrementaldecoder=None, name=None):
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	79 self = tuple.__new__(cls, (encode, decode, streamreader, streamwriter))
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	80 self.name = name
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	81 self.encode = encode
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	82 self.decode = decode
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	83 self.incrementalencoder = incrementalencoder
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	84 self.incrementaldecoder = incrementaldecoder
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	85 self.streamwriter = streamwriter
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	86 self.streamreader = streamreader
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	87 return self
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	88
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	89 def __repr__(self):
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	90 return "<%s.%s object for encoding %s at 0x%x>" % (self.__class__.__module__, self.__class__.__name__, self.name, id(self))
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	91
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	92 class Codec:
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	93
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	94 """ Defines the interface for stateless encoders/decoders.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	95
26293 ff5b510eba6b Add missing documentation for the PEP 293 functionality to Walter Dörwald <walter@livinglogic.de> parents: 26082 diff changeset	96 The .encode()/.decode() methods may use different error
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	97 handling schemes by providing the errors argument. These
26293 ff5b510eba6b Add missing documentation for the PEP 293 functionality to Walter Dörwald <walter@livinglogic.de> parents: 26082 diff changeset	98 string values are predefined:
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	99
12051 d89556c04ef9 Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12011 diff changeset	100 'strict' - raise a ValueError error (or a subclass)
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	101 'ignore' - ignore the character and continue with the next
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	102 'replace' - replace with a suitable replacement character;
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	103 Python will use the official U+FFFD REPLACEMENT
26293 ff5b510eba6b Add missing documentation for the PEP 293 functionality to Walter Dörwald <walter@livinglogic.de> parents: 26082 diff changeset	104 CHARACTER for the builtin Unicode codecs on
ff5b510eba6b Add missing documentation for the PEP 293 functionality to Walter Dörwald <walter@livinglogic.de> parents: 26082 diff changeset	105 decoding and '?' on encoding.
ff5b510eba6b Add missing documentation for the PEP 293 functionality to Walter Dörwald <walter@livinglogic.de> parents: 26082 diff changeset	106 'xmlcharrefreplace' - Replace with the appropriate XML
ff5b510eba6b Add missing documentation for the PEP 293 functionality to Walter Dörwald <walter@livinglogic.de> parents: 26082 diff changeset	107 character reference (only for encoding).
ff5b510eba6b Add missing documentation for the PEP 293 functionality to Walter Dörwald <walter@livinglogic.de> parents: 26082 diff changeset	108 'backslashreplace' - Replace with backslashed escape sequences
ff5b510eba6b Add missing documentation for the PEP 293 functionality to Walter Dörwald <walter@livinglogic.de> parents: 26082 diff changeset	109 (only for encoding).
ff5b510eba6b Add missing documentation for the PEP 293 functionality to Walter Dörwald <walter@livinglogic.de> parents: 26082 diff changeset	110
ff5b510eba6b Add missing documentation for the PEP 293 functionality to Walter Dörwald <walter@livinglogic.de> parents: 26082 diff changeset	111 The set of allowed values can be extended via register_error.
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	112
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	113 """
17839 c872fdf10130 Just changed "x,y" to "x, y" everywhere (i.e., inserted horizontal space Tim Peters <tim.peters@gmail.com> parents: 16070 diff changeset	114 def encode(self, input, errors='strict'):
12373 d63b57ac617d Deleted trailing whitespace. This is really a way to be able to add Guido van Rossum <guido@python.org> parents: 12367 diff changeset	115
12010 bd1014215f62 "intput" --> "input" (in docstring); noted by Detlef Lannert Fred Drake <fdrake@acm.org> parents: 11990 diff changeset	116 """ Encodes the object input and returns a tuple (output
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	117 object, length consumed).
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	118
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	119 errors defines the error handling to apply. It defaults to
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	120 'strict' handling.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	121
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	122 The method may not store state in the Codec instance. Use
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	123 StreamCodec for codecs which have to keep state in order to
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	124 make encoding/decoding efficient.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	125
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	126 The encoder must be able to handle zero length input and
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	127 return an empty object of the output object type in this
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	128 situation.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	129
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	130 """
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	131 raise NotImplementedError
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	132
17839 c872fdf10130 Just changed "x,y" to "x, y" everywhere (i.e., inserted horizontal space Tim Peters <tim.peters@gmail.com> parents: 16070 diff changeset	133 def decode(self, input, errors='strict'):
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	134
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	135 """ Decodes the object input and returns a tuple (output
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	136 object, length consumed).
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	137
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	138 input must be an object which provides the bf_getreadbuf
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	139 buffer slot. Python strings, buffer objects and memory
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	140 mapped files are examples of objects providing this slot.
12373 d63b57ac617d Deleted trailing whitespace. This is really a way to be able to add Guido van Rossum <guido@python.org> parents: 12367 diff changeset	141
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	142 errors defines the error handling to apply. It defaults to
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	143 'strict' handling.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	144
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	145 The method may not store state in the Codec instance. Use
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	146 StreamCodec for codecs which have to keep state in order to
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	147 make encoding/decoding efficient.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	148
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	149 The decoder must be able to handle zero length input and
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	150 return an empty object of the output object type in this
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	151 situation.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	152
12373 d63b57ac617d Deleted trailing whitespace. This is really a way to be able to add Guido van Rossum <guido@python.org> parents: 12367 diff changeset	153 """
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	154 raise NotImplementedError
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	155
36996 7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	156 class IncrementalEncoder(object):
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	157 """
37024 f70c036489ec Add some versionadded info to new incremental codec docs and fix doco nits. Neal Norwitz <nnorwitz@gmail.com> parents: 36996 diff changeset	158 An IncrementalEncoder encodes an input in multiple steps. The input can be
36996 7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	159 passed piece by piece to the encode() method. The IncrementalEncoder remembers
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	160 the state of the Encoding process between calls to encode().
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	161 """
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	162 def __init__(self, errors='strict'):
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	163 """
37024 f70c036489ec Add some versionadded info to new incremental codec docs and fix doco nits. Neal Norwitz <nnorwitz@gmail.com> parents: 36996 diff changeset	164 Creates an IncrementalEncoder instance.
36996 7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	165
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	166 The IncrementalEncoder may use different error handling schemes by
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	167 providing the errors keyword argument. See the module docstring
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	168 for a list of possible values.
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	169 """
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	170 self.errors = errors
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	171 self.buffer = ""
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	172
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	173 def encode(self, input, final=False):
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	174 """
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	175 Encodes input and returns the resulting object.
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	176 """
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	177 raise NotImplementedError
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	178
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	179 def reset(self):
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	180 """
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	181 Resets the encoder to the initial state.
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	182 """
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	183
46107 5341b30b1812 Merged revisions 61750,61752,61754,61756,61760,61763,61768,61772,61775,61805,61809,61812,61819,61917,61920,61930,61933-61934 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 40239 diff changeset	184 def getstate(self):
5341b30b1812 Merged revisions 61750,61752,61754,61756,61760,61763,61768,61772,61775,61805,61809,61812,61819,61917,61920,61930,61933-61934 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 40239 diff changeset	185 """
5341b30b1812 Merged revisions 61750,61752,61754,61756,61760,61763,61768,61772,61775,61805,61809,61812,61819,61917,61920,61930,61933-61934 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 40239 diff changeset	186 Return the current state of the encoder.
5341b30b1812 Merged revisions 61750,61752,61754,61756,61760,61763,61768,61772,61775,61805,61809,61812,61819,61917,61920,61930,61933-61934 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 40239 diff changeset	187 """
5341b30b1812 Merged revisions 61750,61752,61754,61756,61760,61763,61768,61772,61775,61805,61809,61812,61819,61917,61920,61930,61933-61934 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 40239 diff changeset	188 return 0
5341b30b1812 Merged revisions 61750,61752,61754,61756,61760,61763,61768,61772,61775,61805,61809,61812,61819,61917,61920,61930,61933-61934 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 40239 diff changeset	189
5341b30b1812 Merged revisions 61750,61752,61754,61756,61760,61763,61768,61772,61775,61805,61809,61812,61819,61917,61920,61930,61933-61934 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 40239 diff changeset	190 def setstate(self, state):
5341b30b1812 Merged revisions 61750,61752,61754,61756,61760,61763,61768,61772,61775,61805,61809,61812,61819,61917,61920,61930,61933-61934 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 40239 diff changeset	191 """
5341b30b1812 Merged revisions 61750,61752,61754,61756,61760,61763,61768,61772,61775,61805,61809,61812,61819,61917,61920,61930,61933-61934 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 40239 diff changeset	192 Set the current state of the encoder. state must have been
5341b30b1812 Merged revisions 61750,61752,61754,61756,61760,61763,61768,61772,61775,61805,61809,61812,61819,61917,61920,61930,61933-61934 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 40239 diff changeset	193 returned by getstate().
5341b30b1812 Merged revisions 61750,61752,61754,61756,61760,61763,61768,61772,61775,61805,61809,61812,61819,61917,61920,61930,61933-61934 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 40239 diff changeset	194 """
5341b30b1812 Merged revisions 61750,61752,61754,61756,61760,61763,61768,61772,61775,61805,61809,61812,61819,61917,61920,61930,61933-61934 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 40239 diff changeset	195
37719 cf71c3814b0b Add a BufferedIncrementalEncoder class that can be used for implementing Walter Dörwald <walter@livinglogic.de> parents: 37716 diff changeset	196 class BufferedIncrementalEncoder(IncrementalEncoder):
cf71c3814b0b Add a BufferedIncrementalEncoder class that can be used for implementing Walter Dörwald <walter@livinglogic.de> parents: 37716 diff changeset	197 """
cf71c3814b0b Add a BufferedIncrementalEncoder class that can be used for implementing Walter Dörwald <walter@livinglogic.de> parents: 37716 diff changeset	198 This subclass of IncrementalEncoder can be used as the baseclass for an
cf71c3814b0b Add a BufferedIncrementalEncoder class that can be used for implementing Walter Dörwald <walter@livinglogic.de> parents: 37716 diff changeset	199 incremental encoder if the encoder must keep some of the output in a
cf71c3814b0b Add a BufferedIncrementalEncoder class that can be used for implementing Walter Dörwald <walter@livinglogic.de> parents: 37716 diff changeset	200 buffer between calls to encode().
cf71c3814b0b Add a BufferedIncrementalEncoder class that can be used for implementing Walter Dörwald <walter@livinglogic.de> parents: 37716 diff changeset	201 """
cf71c3814b0b Add a BufferedIncrementalEncoder class that can be used for implementing Walter Dörwald <walter@livinglogic.de> parents: 37716 diff changeset	202 def __init__(self, errors='strict'):
cf71c3814b0b Add a BufferedIncrementalEncoder class that can be used for implementing Walter Dörwald <walter@livinglogic.de> parents: 37716 diff changeset	203 IncrementalEncoder.__init__(self, errors)
cf71c3814b0b Add a BufferedIncrementalEncoder class that can be used for implementing Walter Dörwald <walter@livinglogic.de> parents: 37716 diff changeset	204 self.buffer = "" # unencoded input that is kept between calls to encode()
cf71c3814b0b Add a BufferedIncrementalEncoder class that can be used for implementing Walter Dörwald <walter@livinglogic.de> parents: 37716 diff changeset	205
cf71c3814b0b Add a BufferedIncrementalEncoder class that can be used for implementing Walter Dörwald <walter@livinglogic.de> parents: 37716 diff changeset	206 def _buffer_encode(self, input, errors, final):
cf71c3814b0b Add a BufferedIncrementalEncoder class that can be used for implementing Walter Dörwald <walter@livinglogic.de> parents: 37716 diff changeset	207 # Overwrite this method in subclasses: It must encode input
cf71c3814b0b Add a BufferedIncrementalEncoder class that can be used for implementing Walter Dörwald <walter@livinglogic.de> parents: 37716 diff changeset	208 # and return an (output, length consumed) tuple
cf71c3814b0b Add a BufferedIncrementalEncoder class that can be used for implementing Walter Dörwald <walter@livinglogic.de> parents: 37716 diff changeset	209 raise NotImplementedError
cf71c3814b0b Add a BufferedIncrementalEncoder class that can be used for implementing Walter Dörwald <walter@livinglogic.de> parents: 37716 diff changeset	210
cf71c3814b0b Add a BufferedIncrementalEncoder class that can be used for implementing Walter Dörwald <walter@livinglogic.de> parents: 37716 diff changeset	211 def encode(self, input, final=False):
cf71c3814b0b Add a BufferedIncrementalEncoder class that can be used for implementing Walter Dörwald <walter@livinglogic.de> parents: 37716 diff changeset	212 # encode input (taking the buffer into account)
cf71c3814b0b Add a BufferedIncrementalEncoder class that can be used for implementing Walter Dörwald <walter@livinglogic.de> parents: 37716 diff changeset	213 data = self.buffer + input
cf71c3814b0b Add a BufferedIncrementalEncoder class that can be used for implementing Walter Dörwald <walter@livinglogic.de> parents: 37716 diff changeset	214 (result, consumed) = self._buffer_encode(data, self.errors, final)
cf71c3814b0b Add a BufferedIncrementalEncoder class that can be used for implementing Walter Dörwald <walter@livinglogic.de> parents: 37716 diff changeset	215 # keep unencoded input until the next call
cf71c3814b0b Add a BufferedIncrementalEncoder class that can be used for implementing Walter Dörwald <walter@livinglogic.de> parents: 37716 diff changeset	216 self.buffer = data[consumed:]
cf71c3814b0b Add a BufferedIncrementalEncoder class that can be used for implementing Walter Dörwald <walter@livinglogic.de> parents: 37716 diff changeset	217 return result
cf71c3814b0b Add a BufferedIncrementalEncoder class that can be used for implementing Walter Dörwald <walter@livinglogic.de> parents: 37716 diff changeset	218
cf71c3814b0b Add a BufferedIncrementalEncoder class that can be used for implementing Walter Dörwald <walter@livinglogic.de> parents: 37716 diff changeset	219 def reset(self):
cf71c3814b0b Add a BufferedIncrementalEncoder class that can be used for implementing Walter Dörwald <walter@livinglogic.de> parents: 37716 diff changeset	220 IncrementalEncoder.reset(self)
cf71c3814b0b Add a BufferedIncrementalEncoder class that can be used for implementing Walter Dörwald <walter@livinglogic.de> parents: 37716 diff changeset	221 self.buffer = ""
cf71c3814b0b Add a BufferedIncrementalEncoder class that can be used for implementing Walter Dörwald <walter@livinglogic.de> parents: 37716 diff changeset	222
46107 5341b30b1812 Merged revisions 61750,61752,61754,61756,61760,61763,61768,61772,61775,61805,61809,61812,61819,61917,61920,61930,61933-61934 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 40239 diff changeset	223 def getstate(self):
5341b30b1812 Merged revisions 61750,61752,61754,61756,61760,61763,61768,61772,61775,61805,61809,61812,61819,61917,61920,61930,61933-61934 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 40239 diff changeset	224 return self.buffer or 0
5341b30b1812 Merged revisions 61750,61752,61754,61756,61760,61763,61768,61772,61775,61805,61809,61812,61819,61917,61920,61930,61933-61934 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 40239 diff changeset	225
5341b30b1812 Merged revisions 61750,61752,61754,61756,61760,61763,61768,61772,61775,61805,61809,61812,61819,61917,61920,61930,61933-61934 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 40239 diff changeset	226 def setstate(self, state):
5341b30b1812 Merged revisions 61750,61752,61754,61756,61760,61763,61768,61772,61775,61805,61809,61812,61819,61917,61920,61930,61933-61934 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 40239 diff changeset	227 self.buffer = state or ""
5341b30b1812 Merged revisions 61750,61752,61754,61756,61760,61763,61768,61772,61775,61805,61809,61812,61819,61917,61920,61930,61933-61934 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 40239 diff changeset	228
36996 7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	229 class IncrementalDecoder(object):
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	230 """
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	231 An IncrementalDecoder decodes an input in multiple steps. The input can be
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	232 passed piece by piece to the decode() method. The IncrementalDecoder
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	233 remembers the state of the decoding process between calls to decode().
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	234 """
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	235 def __init__(self, errors='strict'):
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	236 """
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	237 Creates a IncrementalDecoder instance.
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	238
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	239 The IncrementalDecoder may use different error handling schemes by
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	240 providing the errors keyword argument. See the module docstring
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	241 for a list of possible values.
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	242 """
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	243 self.errors = errors
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	244
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	245 def decode(self, input, final=False):
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	246 """
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	247 Decodes input and returns the resulting object.
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	248 """
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	249 raise NotImplementedError
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	250
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	251 def reset(self):
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	252 """
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	253 Resets the decoder to the initial state.
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	254 """
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	255
46107 5341b30b1812 Merged revisions 61750,61752,61754,61756,61760,61763,61768,61772,61775,61805,61809,61812,61819,61917,61920,61930,61933-61934 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 40239 diff changeset	256 def getstate(self):
5341b30b1812 Merged revisions 61750,61752,61754,61756,61760,61763,61768,61772,61775,61805,61809,61812,61819,61917,61920,61930,61933-61934 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 40239 diff changeset	257 """
5341b30b1812 Merged revisions 61750,61752,61754,61756,61760,61763,61768,61772,61775,61805,61809,61812,61819,61917,61920,61930,61933-61934 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 40239 diff changeset	258 Return the current state of the decoder.
5341b30b1812 Merged revisions 61750,61752,61754,61756,61760,61763,61768,61772,61775,61805,61809,61812,61819,61917,61920,61930,61933-61934 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 40239 diff changeset	259
5341b30b1812 Merged revisions 61750,61752,61754,61756,61760,61763,61768,61772,61775,61805,61809,61812,61819,61917,61920,61930,61933-61934 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 40239 diff changeset	260 This must be a (buffered_input, additional_state_info) tuple.
5341b30b1812 Merged revisions 61750,61752,61754,61756,61760,61763,61768,61772,61775,61805,61809,61812,61819,61917,61920,61930,61933-61934 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 40239 diff changeset	261 buffered_input must be a bytes object containing bytes that
5341b30b1812 Merged revisions 61750,61752,61754,61756,61760,61763,61768,61772,61775,61805,61809,61812,61819,61917,61920,61930,61933-61934 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 40239 diff changeset	262 were passed to decode() that have not yet been converted.
5341b30b1812 Merged revisions 61750,61752,61754,61756,61760,61763,61768,61772,61775,61805,61809,61812,61819,61917,61920,61930,61933-61934 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 40239 diff changeset	263 additional_state_info must be a non-negative integer
5341b30b1812 Merged revisions 61750,61752,61754,61756,61760,61763,61768,61772,61775,61805,61809,61812,61819,61917,61920,61930,61933-61934 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 40239 diff changeset	264 representing the state of the decoder WITHOUT yet having
5341b30b1812 Merged revisions 61750,61752,61754,61756,61760,61763,61768,61772,61775,61805,61809,61812,61819,61917,61920,61930,61933-61934 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 40239 diff changeset	265 processed the contents of buffered_input. In the initial state
5341b30b1812 Merged revisions 61750,61752,61754,61756,61760,61763,61768,61772,61775,61805,61809,61812,61819,61917,61920,61930,61933-61934 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 40239 diff changeset	266 and after reset(), getstate() must return (b"", 0).
5341b30b1812 Merged revisions 61750,61752,61754,61756,61760,61763,61768,61772,61775,61805,61809,61812,61819,61917,61920,61930,61933-61934 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 40239 diff changeset	267 """
5341b30b1812 Merged revisions 61750,61752,61754,61756,61760,61763,61768,61772,61775,61805,61809,61812,61819,61917,61920,61930,61933-61934 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 40239 diff changeset	268 return (b"", 0)
5341b30b1812 Merged revisions 61750,61752,61754,61756,61760,61763,61768,61772,61775,61805,61809,61812,61819,61917,61920,61930,61933-61934 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 40239 diff changeset	269
5341b30b1812 Merged revisions 61750,61752,61754,61756,61760,61763,61768,61772,61775,61805,61809,61812,61819,61917,61920,61930,61933-61934 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 40239 diff changeset	270 def setstate(self, state):
5341b30b1812 Merged revisions 61750,61752,61754,61756,61760,61763,61768,61772,61775,61805,61809,61812,61819,61917,61920,61930,61933-61934 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 40239 diff changeset	271 """
5341b30b1812 Merged revisions 61750,61752,61754,61756,61760,61763,61768,61772,61775,61805,61809,61812,61819,61917,61920,61930,61933-61934 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 40239 diff changeset	272 Set the current state of the decoder.
5341b30b1812 Merged revisions 61750,61752,61754,61756,61760,61763,61768,61772,61775,61805,61809,61812,61819,61917,61920,61930,61933-61934 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 40239 diff changeset	273
5341b30b1812 Merged revisions 61750,61752,61754,61756,61760,61763,61768,61772,61775,61805,61809,61812,61819,61917,61920,61930,61933-61934 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 40239 diff changeset	274 state must have been returned by getstate(). The effect of
5341b30b1812 Merged revisions 61750,61752,61754,61756,61760,61763,61768,61772,61775,61805,61809,61812,61819,61917,61920,61930,61933-61934 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 40239 diff changeset	275 setstate((b"", 0)) must be equivalent to reset().
5341b30b1812 Merged revisions 61750,61752,61754,61756,61760,61763,61768,61772,61775,61805,61809,61812,61819,61917,61920,61930,61933-61934 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 40239 diff changeset	276 """
5341b30b1812 Merged revisions 61750,61752,61754,61756,61760,61763,61768,61772,61775,61805,61809,61812,61819,61917,61920,61930,61933-61934 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 40239 diff changeset	277
36996 7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	278 class BufferedIncrementalDecoder(IncrementalDecoder):
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	279 """
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	280 This subclass of IncrementalDecoder can be used as the baseclass for an
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	281 incremental decoder if the decoder must be able to handle incomplete byte
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	282 sequences.
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	283 """
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	284 def __init__(self, errors='strict'):
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	285 IncrementalDecoder.__init__(self, errors)
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	286 self.buffer = "" # undecoded input that is kept between calls to decode()
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	287
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	288 def _buffer_decode(self, input, errors, final):
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	289 # Overwrite this method in subclasses: It must decode input
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	290 # and return an (output, length consumed) tuple
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	291 raise NotImplementedError
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	292
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	293 def decode(self, input, final=False):
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	294 # decode input (taking the buffer into account)
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	295 data = self.buffer + input
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	296 (result, consumed) = self._buffer_decode(data, self.errors, final)
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	297 # keep undecoded input until the next call
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	298 self.buffer = data[consumed:]
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	299 return result
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	300
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	301 def reset(self):
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	302 IncrementalDecoder.reset(self)
37716 1d5a4c82061e Fix wrong attribute name. Walter Dörwald <walter@livinglogic.de> parents: 37080 diff changeset	303 self.buffer = ""
36996 7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	304
46107 5341b30b1812 Merged revisions 61750,61752,61754,61756,61760,61763,61768,61772,61775,61805,61809,61812,61819,61917,61920,61930,61933-61934 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 40239 diff changeset	305 def getstate(self):
5341b30b1812 Merged revisions 61750,61752,61754,61756,61760,61763,61768,61772,61775,61805,61809,61812,61819,61917,61920,61930,61933-61934 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 40239 diff changeset	306 # additional state info is always 0
5341b30b1812 Merged revisions 61750,61752,61754,61756,61760,61763,61768,61772,61775,61805,61809,61812,61819,61917,61920,61930,61933-61934 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 40239 diff changeset	307 return (self.buffer, 0)
5341b30b1812 Merged revisions 61750,61752,61754,61756,61760,61763,61768,61772,61775,61805,61809,61812,61819,61917,61920,61930,61933-61934 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 40239 diff changeset	308
5341b30b1812 Merged revisions 61750,61752,61754,61756,61760,61763,61768,61772,61775,61805,61809,61812,61819,61917,61920,61930,61933-61934 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 40239 diff changeset	309 def setstate(self, state):
5341b30b1812 Merged revisions 61750,61752,61754,61756,61760,61763,61768,61772,61775,61805,61809,61812,61819,61917,61920,61930,61933-61934 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 40239 diff changeset	310 # ignore additional state info
5341b30b1812 Merged revisions 61750,61752,61754,61756,61760,61763,61768,61772,61775,61805,61809,61812,61819,61917,61920,61930,61933-61934 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 40239 diff changeset	311 self.buffer = state[0]
5341b30b1812 Merged revisions 61750,61752,61754,61756,61760,61763,61768,61772,61775,61805,61809,61812,61819,61917,61920,61930,61933-61934 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 40239 diff changeset	312
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	313 #
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	314 # The StreamWriter and StreamReader class provide generic working
19967 0b51e607f333 Fix typo in comment Andrew M. Kuchling <amk@amk.ca> parents: 18936 diff changeset	315 # interfaces which can be used to implement new encoding submodules
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	316 # very easily. See encodings/utf_8.py for an example on how this is
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	317 # done.
12373 d63b57ac617d Deleted trailing whitespace. This is really a way to be able to add Guido van Rossum <guido@python.org> parents: 12367 diff changeset	318 #
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	319
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	320 class StreamWriter(Codec):
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	321
17839 c872fdf10130 Just changed "x,y" to "x, y" everywhere (i.e., inserted horizontal space Tim Peters <tim.peters@gmail.com> parents: 16070 diff changeset	322 def __init__(self, stream, errors='strict'):
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	323
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	324 """ Creates a StreamWriter instance.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	325
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	326 stream must be a file-like object open for writing
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	327 (binary) data.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	328
26293 ff5b510eba6b Add missing documentation for the PEP 293 functionality to Walter Dörwald <walter@livinglogic.de> parents: 26082 diff changeset	329 The StreamWriter may use different error handling
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	330 schemes by providing the errors keyword argument. These
26293 ff5b510eba6b Add missing documentation for the PEP 293 functionality to Walter Dörwald <walter@livinglogic.de> parents: 26082 diff changeset	331 parameters are predefined:
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	332
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	333 'strict' - raise a ValueError (or a subclass)
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	334 'ignore' - ignore the character and continue with the next
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	335 'replace'- replace with a suitable replacement character
26293 ff5b510eba6b Add missing documentation for the PEP 293 functionality to Walter Dörwald <walter@livinglogic.de> parents: 26082 diff changeset	336 'xmlcharrefreplace' - Replace with the appropriate XML
ff5b510eba6b Add missing documentation for the PEP 293 functionality to Walter Dörwald <walter@livinglogic.de> parents: 26082 diff changeset	337 character reference.
ff5b510eba6b Add missing documentation for the PEP 293 functionality to Walter Dörwald <walter@livinglogic.de> parents: 26082 diff changeset	338 'backslashreplace' - Replace with backslashed escape
ff5b510eba6b Add missing documentation for the PEP 293 functionality to Walter Dörwald <walter@livinglogic.de> parents: 26082 diff changeset	339 sequences (only for encoding).
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	340
26293 ff5b510eba6b Add missing documentation for the PEP 293 functionality to Walter Dörwald <walter@livinglogic.de> parents: 26082 diff changeset	341 The set of allowed parameter values can be extended via
ff5b510eba6b Add missing documentation for the PEP 293 functionality to Walter Dörwald <walter@livinglogic.de> parents: 26082 diff changeset	342 register_error.
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	343 """
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	344 self.stream = stream
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	345 self.errors = errors
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	346
12367 677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	347 def write(self, object):
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	348
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	349 """ Writes the object's contents encoded to self.stream.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	350 """
17839 c872fdf10130 Just changed "x,y" to "x, y" everywhere (i.e., inserted horizontal space Tim Peters <tim.peters@gmail.com> parents: 16070 diff changeset	351 data, consumed = self.encode(object, self.errors)
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	352 self.stream.write(data)
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	353
12367 677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	354 def writelines(self, list):
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	355
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	356 """ Writes the concatenated list of strings to the stream
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	357 using .write().
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	358 """
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	359 self.write(''.join(list))
12373 d63b57ac617d Deleted trailing whitespace. This is really a way to be able to add Guido van Rossum <guido@python.org> parents: 12367 diff changeset	360
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	361 def reset(self):
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	362
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	363 """ Flushes and resets the codec buffers used for keeping state.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	364
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	365 Calling this method should ensure that the data on the
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	366 output is put into a clean state, that allows appending
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	367 of new fresh data without having to rescan the whole
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	368 stream to recover state.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	369
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	370 """
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	371 pass
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	372
17839 c872fdf10130 Just changed "x,y" to "x, y" everywhere (i.e., inserted horizontal space Tim Peters <tim.peters@gmail.com> parents: 16070 diff changeset	373 def __getattr__(self, name,
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	374 getattr=getattr):
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	375
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	376 """ Inherit all other methods from the underlying stream.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	377 """
17839 c872fdf10130 Just changed "x,y" to "x, y" everywhere (i.e., inserted horizontal space Tim Peters <tim.peters@gmail.com> parents: 16070 diff changeset	378 return getattr(self.stream, name)
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	379
40239 7c4058681224 Fix codecs.EncodedFile which did not use file_encoding in 2.5.0, and Georg Brandl <georg@python.org> parents: 37719 diff changeset	380 def __enter__(self):
7c4058681224 Fix codecs.EncodedFile which did not use file_encoding in 2.5.0, and Georg Brandl <georg@python.org> parents: 37719 diff changeset	381 return self
7c4058681224 Fix codecs.EncodedFile which did not use file_encoding in 2.5.0, and Georg Brandl <georg@python.org> parents: 37719 diff changeset	382
7c4058681224 Fix codecs.EncodedFile which did not use file_encoding in 2.5.0, and Georg Brandl <georg@python.org> parents: 37719 diff changeset	383 def __exit__(self, type, value, tb):
7c4058681224 Fix codecs.EncodedFile which did not use file_encoding in 2.5.0, and Georg Brandl <georg@python.org> parents: 37719 diff changeset	384 self.stream.close()
7c4058681224 Fix codecs.EncodedFile which did not use file_encoding in 2.5.0, and Georg Brandl <georg@python.org> parents: 37719 diff changeset	385
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	386 ###
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	387
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	388 class StreamReader(Codec):
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	389
17839 c872fdf10130 Just changed "x,y" to "x, y" everywhere (i.e., inserted horizontal space Tim Peters <tim.peters@gmail.com> parents: 16070 diff changeset	390 def __init__(self, stream, errors='strict'):
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	391
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	392 """ Creates a StreamReader instance.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	393
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	394 stream must be a file-like object open for reading
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	395 (binary) data.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	396
26293 ff5b510eba6b Add missing documentation for the PEP 293 functionality to Walter Dörwald <walter@livinglogic.de> parents: 26082 diff changeset	397 The StreamReader may use different error handling
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	398 schemes by providing the errors keyword argument. These
26293 ff5b510eba6b Add missing documentation for the PEP 293 functionality to Walter Dörwald <walter@livinglogic.de> parents: 26082 diff changeset	399 parameters are predefined:
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	400
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	401 'strict' - raise a ValueError (or a subclass)
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	402 'ignore' - ignore the character and continue with the next
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	403 'replace'- replace with a suitable replacement character;
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	404
26293 ff5b510eba6b Add missing documentation for the PEP 293 functionality to Walter Dörwald <walter@livinglogic.de> parents: 26082 diff changeset	405 The set of allowed parameter values can be extended via
ff5b510eba6b Add missing documentation for the PEP 293 functionality to Walter Dörwald <walter@livinglogic.de> parents: 26082 diff changeset	406 register_error.
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	407 """
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	408 self.stream = stream
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	409 self.errors = errors
33428 57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 31591 diff changeset	410 self.bytebuffer = ""
35269 2b117eb18944 Make attributes and local variables in the StreamReader str objects instead Walter Dörwald <walter@livinglogic.de> parents: 34924 diff changeset	411 # For str->str decoding this will stay a str
2b117eb18944 Make attributes and local variables in the StreamReader str objects instead Walter Dörwald <walter@livinglogic.de> parents: 34924 diff changeset	412 # For str->unicode decoding the first read will promote it to unicode
2b117eb18944 Make attributes and local variables in the StreamReader str objects instead Walter Dörwald <walter@livinglogic.de> parents: 34924 diff changeset	413 self.charbuffer = ""
35589 5f2785722cba Patch #1268314: Cache lines in StreamReader.readlines for performance. Martin v. Löwis <martin@v.loewis.de> parents: 35508 diff changeset	414 self.linebuffer = None
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	415
33428 57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 31591 diff changeset	416 def decode(self, input, errors='strict'):
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 31591 diff changeset	417 raise NotImplementedError
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 31591 diff changeset	418
35415 ff2e66fc0a29 Return complete lines from codec stream readers Martin v. Löwis <martin@v.loewis.de> parents: 35269 diff changeset	419 def read(self, size=-1, chars=-1, firstline=False):
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	420
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	421 """ Decodes data from the stream self.stream and returns the
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	422 resulting object.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	423
33428 57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 31591 diff changeset	424 chars indicates the number of characters to read from the
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 31591 diff changeset	425 stream. read() will never return more than chars
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 31591 diff changeset	426 characters, but it might return less, if there are not enough
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 31591 diff changeset	427 characters available.
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 31591 diff changeset	428
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	429 size indicates the approximate maximum number of bytes to
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	430 read from the stream for decoding purposes. The decoder
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	431 can modify this setting as appropriate. The default value
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	432 -1 indicates to read and decode as much as possible. size
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	433 is intended to prevent having to decode huge files in one
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	434 step.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	435
35415 ff2e66fc0a29 Return complete lines from codec stream readers Martin v. Löwis <martin@v.loewis.de> parents: 35269 diff changeset	436 If firstline is true, and a UnicodeDecodeError happens
ff2e66fc0a29 Return complete lines from codec stream readers Martin v. Löwis <martin@v.loewis.de> parents: 35269 diff changeset	437 after the first line terminator in the input only the first line
ff2e66fc0a29 Return complete lines from codec stream readers Martin v. Löwis <martin@v.loewis.de> parents: 35269 diff changeset	438 will be returned, the rest of the input will be kept until the
ff2e66fc0a29 Return complete lines from codec stream readers Martin v. Löwis <martin@v.loewis.de> parents: 35269 diff changeset	439 next call to read().
ff2e66fc0a29 Return complete lines from codec stream readers Martin v. Löwis <martin@v.loewis.de> parents: 35269 diff changeset	440
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	441 The method should use a greedy read strategy meaning that
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	442 it should read as much data as is allowed within the
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	443 definition of the encoding and the given size, e.g. if
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	444 optional encoding endings or state markers are available
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	445 on the stream, these should be read too.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	446 """
35589 5f2785722cba Patch #1268314: Cache lines in StreamReader.readlines for performance. Martin v. Löwis <martin@v.loewis.de> parents: 35508 diff changeset	447 # If we have lines cached, first merge them back into characters
5f2785722cba Patch #1268314: Cache lines in StreamReader.readlines for performance. Martin v. Löwis <martin@v.loewis.de> parents: 35508 diff changeset	448 if self.linebuffer:
5f2785722cba Patch #1268314: Cache lines in StreamReader.readlines for performance. Martin v. Löwis <martin@v.loewis.de> parents: 35508 diff changeset	449 self.charbuffer = "".join(self.linebuffer)
5f2785722cba Patch #1268314: Cache lines in StreamReader.readlines for performance. Martin v. Löwis <martin@v.loewis.de> parents: 35508 diff changeset	450 self.linebuffer = None
36121 686306a67e9b Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 35589 diff changeset	451
33428 57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 31591 diff changeset	452 # read until we get the required number of characters (if available)
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 31591 diff changeset	453 while True:
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 31591 diff changeset	454 # can the request can be satisfied from the character buffer?
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 31591 diff changeset	455 if chars < 0:
36855 204c307b01ce If size is specified, try to read at least size characters. Walter Dörwald <walter@livinglogic.de> parents: 36121 diff changeset	456 if size < 0:
204c307b01ce If size is specified, try to read at least size characters. Walter Dörwald <walter@livinglogic.de> parents: 36121 diff changeset	457 if self.charbuffer:
204c307b01ce If size is specified, try to read at least size characters. Walter Dörwald <walter@livinglogic.de> parents: 36121 diff changeset	458 break
204c307b01ce If size is specified, try to read at least size characters. Walter Dörwald <walter@livinglogic.de> parents: 36121 diff changeset	459 elif len(self.charbuffer) >= size:
34214 abe3145dde4e Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 33782 diff changeset	460 break
33428 57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 31591 diff changeset	461 else:
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 31591 diff changeset	462 if len(self.charbuffer) >= chars:
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 31591 diff changeset	463 break
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 31591 diff changeset	464 # we need more data
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 31591 diff changeset	465 if size < 0:
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 31591 diff changeset	466 newdata = self.stream.read()
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 31591 diff changeset	467 else:
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 31591 diff changeset	468 newdata = self.stream.read(size)
34214 abe3145dde4e Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 33782 diff changeset	469 # decode bytes (those remaining from the last call included)
33428 57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 31591 diff changeset	470 data = self.bytebuffer + newdata
35415 ff2e66fc0a29 Return complete lines from codec stream readers Martin v. Löwis <martin@v.loewis.de> parents: 35269 diff changeset	471 try:
ff2e66fc0a29 Return complete lines from codec stream readers Martin v. Löwis <martin@v.loewis.de> parents: 35269 diff changeset	472 newchars, decodedbytes = self.decode(data, self.errors)
ff2e66fc0a29 Return complete lines from codec stream readers Martin v. Löwis <martin@v.loewis.de> parents: 35269 diff changeset	473 except UnicodeDecodeError, exc:
ff2e66fc0a29 Return complete lines from codec stream readers Martin v. Löwis <martin@v.loewis.de> parents: 35269 diff changeset	474 if firstline:
ff2e66fc0a29 Return complete lines from codec stream readers Martin v. Löwis <martin@v.loewis.de> parents: 35269 diff changeset	475 newchars, decodedbytes = self.decode(data[:exc.start], self.errors)
ff2e66fc0a29 Return complete lines from codec stream readers Martin v. Löwis <martin@v.loewis.de> parents: 35269 diff changeset	476 lines = newchars.splitlines(True)
ff2e66fc0a29 Return complete lines from codec stream readers Martin v. Löwis <martin@v.loewis.de> parents: 35269 diff changeset	477 if len(lines)<=1:
ff2e66fc0a29 Return complete lines from codec stream readers Martin v. Löwis <martin@v.loewis.de> parents: 35269 diff changeset	478 raise
ff2e66fc0a29 Return complete lines from codec stream readers Martin v. Löwis <martin@v.loewis.de> parents: 35269 diff changeset	479 else:
ff2e66fc0a29 Return complete lines from codec stream readers Martin v. Löwis <martin@v.loewis.de> parents: 35269 diff changeset	480 raise
33428 57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 31591 diff changeset	481 # keep undecoded bytes until the next call
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 31591 diff changeset	482 self.bytebuffer = data[decodedbytes:]
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 31591 diff changeset	483 # put new characters in the character buffer
34214 abe3145dde4e Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 33782 diff changeset	484 self.charbuffer += newchars
33428 57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 31591 diff changeset	485 # there was no data available
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 31591 diff changeset	486 if not newdata:
34214 abe3145dde4e Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 33782 diff changeset	487 break
abe3145dde4e Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 33782 diff changeset	488 if chars < 0:
abe3145dde4e Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 33782 diff changeset	489 # Return everything we've got
abe3145dde4e Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 33782 diff changeset	490 result = self.charbuffer
35269 2b117eb18944 Make attributes and local variables in the StreamReader str objects instead Walter Dörwald <walter@livinglogic.de> parents: 34924 diff changeset	491 self.charbuffer = ""
34214 abe3145dde4e Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 33782 diff changeset	492 else:
abe3145dde4e Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 33782 diff changeset	493 # Return the first chars characters
abe3145dde4e Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 33782 diff changeset	494 result = self.charbuffer[:chars]
abe3145dde4e Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 33782 diff changeset	495 self.charbuffer = self.charbuffer[chars:]
33428 57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 31591 diff changeset	496 return result
12373 d63b57ac617d Deleted trailing whitespace. This is really a way to be able to add Guido van Rossum <guido@python.org> parents: 12367 diff changeset	497
33428 57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 31591 diff changeset	498 def readline(self, size=None, keepends=True):
12367 677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	499
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	500 """ Read one line from the input stream and return the
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	501 decoded data.
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	502
33428 57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 31591 diff changeset	503 size, if given, is passed as size argument to the
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 31591 diff changeset	504 read() method.
12373 d63b57ac617d Deleted trailing whitespace. This is really a way to be able to add Guido van Rossum <guido@python.org> parents: 12367 diff changeset	505
12367 677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	506 """
35589 5f2785722cba Patch #1268314: Cache lines in StreamReader.readlines for performance. Martin v. Löwis <martin@v.loewis.de> parents: 35508 diff changeset	507 # If we have lines cached from an earlier read, return
5f2785722cba Patch #1268314: Cache lines in StreamReader.readlines for performance. Martin v. Löwis <martin@v.loewis.de> parents: 35508 diff changeset	508 # them unconditionally
5f2785722cba Patch #1268314: Cache lines in StreamReader.readlines for performance. Martin v. Löwis <martin@v.loewis.de> parents: 35508 diff changeset	509 if self.linebuffer:
5f2785722cba Patch #1268314: Cache lines in StreamReader.readlines for performance. Martin v. Löwis <martin@v.loewis.de> parents: 35508 diff changeset	510 line = self.linebuffer[0]
5f2785722cba Patch #1268314: Cache lines in StreamReader.readlines for performance. Martin v. Löwis <martin@v.loewis.de> parents: 35508 diff changeset	511 del self.linebuffer[0]
5f2785722cba Patch #1268314: Cache lines in StreamReader.readlines for performance. Martin v. Löwis <martin@v.loewis.de> parents: 35508 diff changeset	512 if len(self.linebuffer) == 1:
5f2785722cba Patch #1268314: Cache lines in StreamReader.readlines for performance. Martin v. Löwis <martin@v.loewis.de> parents: 35508 diff changeset	513 # revert to charbuffer mode; we might need more data
5f2785722cba Patch #1268314: Cache lines in StreamReader.readlines for performance. Martin v. Löwis <martin@v.loewis.de> parents: 35508 diff changeset	514 # next time
5f2785722cba Patch #1268314: Cache lines in StreamReader.readlines for performance. Martin v. Löwis <martin@v.loewis.de> parents: 35508 diff changeset	515 self.charbuffer = self.linebuffer[0]
5f2785722cba Patch #1268314: Cache lines in StreamReader.readlines for performance. Martin v. Löwis <martin@v.loewis.de> parents: 35508 diff changeset	516 self.linebuffer = None
5f2785722cba Patch #1268314: Cache lines in StreamReader.readlines for performance. Martin v. Löwis <martin@v.loewis.de> parents: 35508 diff changeset	517 if not keepends:
5f2785722cba Patch #1268314: Cache lines in StreamReader.readlines for performance. Martin v. Löwis <martin@v.loewis.de> parents: 35508 diff changeset	518 line = line.splitlines(False)[0]
5f2785722cba Patch #1268314: Cache lines in StreamReader.readlines for performance. Martin v. Löwis <martin@v.loewis.de> parents: 35508 diff changeset	519 return line
36121 686306a67e9b Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 35589 diff changeset	520
34214 abe3145dde4e Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 33782 diff changeset	521 readsize = size or 72
35269 2b117eb18944 Make attributes and local variables in the StreamReader str objects instead Walter Dörwald <walter@livinglogic.de> parents: 34924 diff changeset	522 line = ""
34214 abe3145dde4e Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 33782 diff changeset	523 # If size is given, we call read() only once
33428 57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 31591 diff changeset	524 while True:
35415 ff2e66fc0a29 Return complete lines from codec stream readers Martin v. Löwis <martin@v.loewis.de> parents: 35269 diff changeset	525 data = self.read(readsize, firstline=True)
34214 abe3145dde4e Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 33782 diff changeset	526 if data:
34924 c66a7fbe16cf Fix comment. Walter Dörwald <walter@livinglogic.de> parents: 34923 diff changeset	527 # If we're at a "\r" read one extra character (which might
c66a7fbe16cf Fix comment. Walter Dörwald <walter@livinglogic.de> parents: 34923 diff changeset	528 # be a "\n") to get a proper line ending. If the stream is
34923 bc123b9cef7f If the data read from the bytestream in readline() ends in a '\r' read one more Walter Dörwald <walter@livinglogic.de> parents: 34866 diff changeset	529 # temporarily exhausted we return the wrong line ending.
35269 2b117eb18944 Make attributes and local variables in the StreamReader str objects instead Walter Dörwald <walter@livinglogic.de> parents: 34924 diff changeset	530 if data.endswith("\r"):
34865 b49d39111c48 Fix for SF bug #1175396: readline() will now read one more character, if Walter Dörwald <walter@livinglogic.de> parents: 34761 diff changeset	531 data += self.read(size=1, chars=1)
b49d39111c48 Fix for SF bug #1175396: readline() will now read one more character, if Walter Dörwald <walter@livinglogic.de> parents: 34761 diff changeset	532
33428 57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 31591 diff changeset	533 line += data
34214 abe3145dde4e Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 33782 diff changeset	534 lines = line.splitlines(True)
abe3145dde4e Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 33782 diff changeset	535 if lines:
35589 5f2785722cba Patch #1268314: Cache lines in StreamReader.readlines for performance. Martin v. Löwis <martin@v.loewis.de> parents: 35508 diff changeset	536 if len(lines) > 1:
5f2785722cba Patch #1268314: Cache lines in StreamReader.readlines for performance. Martin v. Löwis <martin@v.loewis.de> parents: 35508 diff changeset	537 # More than one line result; the first line is a full line
5f2785722cba Patch #1268314: Cache lines in StreamReader.readlines for performance. Martin v. Löwis <martin@v.loewis.de> parents: 35508 diff changeset	538 # to return
5f2785722cba Patch #1268314: Cache lines in StreamReader.readlines for performance. Martin v. Löwis <martin@v.loewis.de> parents: 35508 diff changeset	539 line = lines[0]
5f2785722cba Patch #1268314: Cache lines in StreamReader.readlines for performance. Martin v. Löwis <martin@v.loewis.de> parents: 35508 diff changeset	540 del lines[0]
5f2785722cba Patch #1268314: Cache lines in StreamReader.readlines for performance. Martin v. Löwis <martin@v.loewis.de> parents: 35508 diff changeset	541 if len(lines) > 1:
5f2785722cba Patch #1268314: Cache lines in StreamReader.readlines for performance. Martin v. Löwis <martin@v.loewis.de> parents: 35508 diff changeset	542 # cache the remaining lines
5f2785722cba Patch #1268314: Cache lines in StreamReader.readlines for performance. Martin v. Löwis <martin@v.loewis.de> parents: 35508 diff changeset	543 lines[-1] += self.charbuffer
5f2785722cba Patch #1268314: Cache lines in StreamReader.readlines for performance. Martin v. Löwis <martin@v.loewis.de> parents: 35508 diff changeset	544 self.linebuffer = lines
5f2785722cba Patch #1268314: Cache lines in StreamReader.readlines for performance. Martin v. Löwis <martin@v.loewis.de> parents: 35508 diff changeset	545 self.charbuffer = None
5f2785722cba Patch #1268314: Cache lines in StreamReader.readlines for performance. Martin v. Löwis <martin@v.loewis.de> parents: 35508 diff changeset	546 else:
5f2785722cba Patch #1268314: Cache lines in StreamReader.readlines for performance. Martin v. Löwis <martin@v.loewis.de> parents: 35508 diff changeset	547 # only one remaining line, put it back into charbuffer
5f2785722cba Patch #1268314: Cache lines in StreamReader.readlines for performance. Martin v. Löwis <martin@v.loewis.de> parents: 35508 diff changeset	548 self.charbuffer = lines[0] + self.charbuffer
5f2785722cba Patch #1268314: Cache lines in StreamReader.readlines for performance. Martin v. Löwis <martin@v.loewis.de> parents: 35508 diff changeset	549 if not keepends:
5f2785722cba Patch #1268314: Cache lines in StreamReader.readlines for performance. Martin v. Löwis <martin@v.loewis.de> parents: 35508 diff changeset	550 line = line.splitlines(False)[0]
5f2785722cba Patch #1268314: Cache lines in StreamReader.readlines for performance. Martin v. Löwis <martin@v.loewis.de> parents: 35508 diff changeset	551 break
34214 abe3145dde4e Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 33782 diff changeset	552 line0withend = lines[0]
abe3145dde4e Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 33782 diff changeset	553 line0withoutend = lines[0].splitlines(False)[0]
abe3145dde4e Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 33782 diff changeset	554 if line0withend != line0withoutend: # We really have a line end
abe3145dde4e Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 33782 diff changeset	555 # Put the rest back together and keep it until the next call
35269 2b117eb18944 Make attributes and local variables in the StreamReader str objects instead Walter Dörwald <walter@livinglogic.de> parents: 34924 diff changeset	556 self.charbuffer = "".join(lines[1:]) + self.charbuffer
34214 abe3145dde4e Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 33782 diff changeset	557 if keepends:
abe3145dde4e Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 33782 diff changeset	558 line = line0withend
abe3145dde4e Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 33782 diff changeset	559 else:
abe3145dde4e Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 33782 diff changeset	560 line = line0withoutend
34387 f25dcd7f3e48 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34214 diff changeset	561 break
34214 abe3145dde4e Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 33782 diff changeset	562 # we didn't get anything or this was our only try
34387 f25dcd7f3e48 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34214 diff changeset	563 if not data or size is not None:
34214 abe3145dde4e Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 33782 diff changeset	564 if line and not keepends:
abe3145dde4e Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 33782 diff changeset	565 line = line.splitlines(False)[0]
abe3145dde4e Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 33782 diff changeset	566 break
abe3145dde4e Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 33782 diff changeset	567 if readsize<8000:
abe3145dde4e Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 33782 diff changeset	568 readsize *= 2
abe3145dde4e Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 33782 diff changeset	569 return line
12373 d63b57ac617d Deleted trailing whitespace. This is really a way to be able to add Guido van Rossum <guido@python.org> parents: 12367 diff changeset	570
33428 57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 31591 diff changeset	571 def readlines(self, sizehint=None, keepends=True):
12367 677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	572
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	573 """ Read all lines available on the input stream
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	574 and return them as list of lines.
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	575
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	576 Line breaks are implemented using the codec's decoder
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	577 method and are included in the list entries.
12373 d63b57ac617d Deleted trailing whitespace. This is really a way to be able to add Guido van Rossum <guido@python.org> parents: 12367 diff changeset	578
31591 26a80f0af9f3 Ignore sizehint argument. Fixes SF #844561. Marc-André Lemburg <mal@egenix.com> parents: 27618 diff changeset	579 sizehint, if given, is ignored since there is no efficient
26a80f0af9f3 Ignore sizehint argument. Fixes SF #844561. Marc-André Lemburg <mal@egenix.com> parents: 27618 diff changeset	580 way to finding the true end-of-line.
12367 677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	581
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	582 """
33428 57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support Walter Dörwald <walter@livinglogic.de> parents: 31591 diff changeset	583 data = self.read()
33782 a52de911b284 SF #1048865: Fix a trivial typo that breaks StreamReader.readlines() Hye-Shik Chang <hyeshik@gmail.com> parents: 33428 diff changeset	584 return data.splitlines(keepends)
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	585
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	586 def reset(self):
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	587
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	588 """ Resets the codec buffers used for keeping state.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	589
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	590 Note that no stream repositioning should take place.
13620 5b727ab37092 Spelling fixes supplied by Rob W. W. Hooft. All these are fixes in either Thomas Wouters <thomas@python.org> parents: 12986 diff changeset	591 This method is primarily intended to be able to recover
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	592 from decoding errors.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	593
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	594 """
34751 962800ee30ef Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34707 diff changeset	595 self.bytebuffer = ""
962800ee30ef Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34707 diff changeset	596 self.charbuffer = u""
35589 5f2785722cba Patch #1268314: Cache lines in StreamReader.readlines for performance. Martin v. Löwis <martin@v.loewis.de> parents: 35508 diff changeset	597 self.linebuffer = None
34751 962800ee30ef Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34707 diff changeset	598
34753 11b4350b5879 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34751 diff changeset	599 def seek(self, offset, whence=0):
34751 962800ee30ef Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34707 diff changeset	600 """ Set the input stream's current position.
962800ee30ef Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34707 diff changeset	601
962800ee30ef Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34707 diff changeset	602 Resets the codec buffers used for keeping state.
962800ee30ef Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34707 diff changeset	603 """
962800ee30ef Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34707 diff changeset	604 self.reset()
962800ee30ef Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 34707 diff changeset	605 self.stream.seek(offset, whence)
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	606
26082 f3a64035ba8b Add next() and __iter__() methods to StreamReader, StreamReaderWriter Walter Dörwald <walter@livinglogic.de> parents: 25091 diff changeset	607 def next(self):
f3a64035ba8b Add next() and __iter__() methods to StreamReader, StreamReaderWriter Walter Dörwald <walter@livinglogic.de> parents: 25091 diff changeset	608
f3a64035ba8b Add next() and __iter__() methods to StreamReader, StreamReaderWriter Walter Dörwald <walter@livinglogic.de> parents: 25091 diff changeset	609 """ Return the next decoded line from the input stream."""
f3a64035ba8b Add next() and __iter__() methods to StreamReader, StreamReaderWriter Walter Dörwald <walter@livinglogic.de> parents: 25091 diff changeset	610 line = self.readline()
f3a64035ba8b Add next() and __iter__() methods to StreamReader, StreamReaderWriter Walter Dörwald <walter@livinglogic.de> parents: 25091 diff changeset	611 if line:
f3a64035ba8b Add next() and __iter__() methods to StreamReader, StreamReaderWriter Walter Dörwald <walter@livinglogic.de> parents: 25091 diff changeset	612 return line
f3a64035ba8b Add next() and __iter__() methods to StreamReader, StreamReaderWriter Walter Dörwald <walter@livinglogic.de> parents: 25091 diff changeset	613 raise StopIteration
f3a64035ba8b Add next() and __iter__() methods to StreamReader, StreamReaderWriter Walter Dörwald <walter@livinglogic.de> parents: 25091 diff changeset	614
f3a64035ba8b Add next() and __iter__() methods to StreamReader, StreamReaderWriter Walter Dörwald <walter@livinglogic.de> parents: 25091 diff changeset	615 def __iter__(self):
f3a64035ba8b Add next() and __iter__() methods to StreamReader, StreamReaderWriter Walter Dörwald <walter@livinglogic.de> parents: 25091 diff changeset	616 return self
f3a64035ba8b Add next() and __iter__() methods to StreamReader, StreamReaderWriter Walter Dörwald <walter@livinglogic.de> parents: 25091 diff changeset	617
17839 c872fdf10130 Just changed "x,y" to "x, y" everywhere (i.e., inserted horizontal space Tim Peters <tim.peters@gmail.com> parents: 16070 diff changeset	618 def __getattr__(self, name,
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	619 getattr=getattr):
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	620
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	621 """ Inherit all other methods from the underlying stream.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	622 """
17839 c872fdf10130 Just changed "x,y" to "x, y" everywhere (i.e., inserted horizontal space Tim Peters <tim.peters@gmail.com> parents: 16070 diff changeset	623 return getattr(self.stream, name)
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	624
40239 7c4058681224 Fix codecs.EncodedFile which did not use file_encoding in 2.5.0, and Georg Brandl <georg@python.org> parents: 37719 diff changeset	625 def __enter__(self):
7c4058681224 Fix codecs.EncodedFile which did not use file_encoding in 2.5.0, and Georg Brandl <georg@python.org> parents: 37719 diff changeset	626 return self
7c4058681224 Fix codecs.EncodedFile which did not use file_encoding in 2.5.0, and Georg Brandl <georg@python.org> parents: 37719 diff changeset	627
7c4058681224 Fix codecs.EncodedFile which did not use file_encoding in 2.5.0, and Georg Brandl <georg@python.org> parents: 37719 diff changeset	628 def __exit__(self, type, value, tb):
7c4058681224 Fix codecs.EncodedFile which did not use file_encoding in 2.5.0, and Georg Brandl <georg@python.org> parents: 37719 diff changeset	629 self.stream.close()
7c4058681224 Fix codecs.EncodedFile which did not use file_encoding in 2.5.0, and Georg Brandl <georg@python.org> parents: 37719 diff changeset	630
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	631 ###
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	632
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	633 class StreamReaderWriter:
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	634
12389 e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	635 """ StreamReaderWriter instances allow wrapping streams which
e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	636 work in both read and write modes.
e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	637
e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	638 The design is such that one can use the factory functions
13620 5b727ab37092 Spelling fixes supplied by Rob W. W. Hooft. All these are fixes in either Thomas Wouters <thomas@python.org> parents: 12986 diff changeset	639 returned by the codec.lookup() function to construct the
12389 e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	640 instance.
e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	641
e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	642 """
12367 677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	643 # Optional attributes set by the file wrappers below
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	644 encoding = 'unknown'
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	645
17839 c872fdf10130 Just changed "x,y" to "x, y" everywhere (i.e., inserted horizontal space Tim Peters <tim.peters@gmail.com> parents: 16070 diff changeset	646 def __init__(self, stream, Reader, Writer, errors='strict'):
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	647
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	648 """ Creates a StreamReaderWriter instance.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	649
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	650 stream must be a Stream-like object.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	651
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	652 Reader, Writer must be factory functions or classes
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	653 providing the StreamReader, StreamWriter interface resp.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	654
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	655 Error handling is done in the same way as defined for the
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	656 StreamWriter/Readers.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	657
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	658 """
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	659 self.stream = stream
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	660 self.reader = Reader(stream, errors)
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	661 self.writer = Writer(stream, errors)
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	662 self.errors = errors
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	663
17839 c872fdf10130 Just changed "x,y" to "x, y" everywhere (i.e., inserted horizontal space Tim Peters <tim.peters@gmail.com> parents: 16070 diff changeset	664 def read(self, size=-1):
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	665
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	666 return self.reader.read(size)
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	667
12534 189298a1fab4 Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12389 diff changeset	668 def readline(self, size=None):
12367 677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	669
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	670 return self.reader.readline(size)
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	671
12534 189298a1fab4 Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12389 diff changeset	672 def readlines(self, sizehint=None):
12367 677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	673
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	674 return self.reader.readlines(sizehint)
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	675
26082 f3a64035ba8b Add next() and __iter__() methods to StreamReader, StreamReaderWriter Walter Dörwald <walter@livinglogic.de> parents: 25091 diff changeset	676 def next(self):
f3a64035ba8b Add next() and __iter__() methods to StreamReader, StreamReaderWriter Walter Dörwald <walter@livinglogic.de> parents: 25091 diff changeset	677
f3a64035ba8b Add next() and __iter__() methods to StreamReader, StreamReaderWriter Walter Dörwald <walter@livinglogic.de> parents: 25091 diff changeset	678 """ Return the next decoded line from the input stream."""
f3a64035ba8b Add next() and __iter__() methods to StreamReader, StreamReaderWriter Walter Dörwald <walter@livinglogic.de> parents: 25091 diff changeset	679 return self.reader.next()
f3a64035ba8b Add next() and __iter__() methods to StreamReader, StreamReaderWriter Walter Dörwald <walter@livinglogic.de> parents: 25091 diff changeset	680
f3a64035ba8b Add next() and __iter__() methods to StreamReader, StreamReaderWriter Walter Dörwald <walter@livinglogic.de> parents: 25091 diff changeset	681 def __iter__(self):
f3a64035ba8b Add next() and __iter__() methods to StreamReader, StreamReaderWriter Walter Dörwald <walter@livinglogic.de> parents: 25091 diff changeset	682 return self
f3a64035ba8b Add next() and __iter__() methods to StreamReader, StreamReaderWriter Walter Dörwald <walter@livinglogic.de> parents: 25091 diff changeset	683
17839 c872fdf10130 Just changed "x,y" to "x, y" everywhere (i.e., inserted horizontal space Tim Peters <tim.peters@gmail.com> parents: 16070 diff changeset	684 def write(self, data):
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	685
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	686 return self.writer.write(data)
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	687
17839 c872fdf10130 Just changed "x,y" to "x, y" everywhere (i.e., inserted horizontal space Tim Peters <tim.peters@gmail.com> parents: 16070 diff changeset	688 def writelines(self, list):
12367 677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	689
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	690 return self.writer.writelines(list)
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	691
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	692 def reset(self):
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	693
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	694 self.reader.reset()
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	695 self.writer.reset()
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	696
17839 c872fdf10130 Just changed "x,y" to "x, y" everywhere (i.e., inserted horizontal space Tim Peters <tim.peters@gmail.com> parents: 16070 diff changeset	697 def __getattr__(self, name,
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	698 getattr=getattr):
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	699
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	700 """ Inherit all other methods from the underlying stream.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	701 """
17839 c872fdf10130 Just changed "x,y" to "x, y" everywhere (i.e., inserted horizontal space Tim Peters <tim.peters@gmail.com> parents: 16070 diff changeset	702 return getattr(self.stream, name)
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	703
40239 7c4058681224 Fix codecs.EncodedFile which did not use file_encoding in 2.5.0, and Georg Brandl <georg@python.org> parents: 37719 diff changeset	704 # these are needed to make "with codecs.open(...)" work properly
7c4058681224 Fix codecs.EncodedFile which did not use file_encoding in 2.5.0, and Georg Brandl <georg@python.org> parents: 37719 diff changeset	705
7c4058681224 Fix codecs.EncodedFile which did not use file_encoding in 2.5.0, and Georg Brandl <georg@python.org> parents: 37719 diff changeset	706 def __enter__(self):
7c4058681224 Fix codecs.EncodedFile which did not use file_encoding in 2.5.0, and Georg Brandl <georg@python.org> parents: 37719 diff changeset	707 return self
7c4058681224 Fix codecs.EncodedFile which did not use file_encoding in 2.5.0, and Georg Brandl <georg@python.org> parents: 37719 diff changeset	708
7c4058681224 Fix codecs.EncodedFile which did not use file_encoding in 2.5.0, and Georg Brandl <georg@python.org> parents: 37719 diff changeset	709 def __exit__(self, type, value, tb):
7c4058681224 Fix codecs.EncodedFile which did not use file_encoding in 2.5.0, and Georg Brandl <georg@python.org> parents: 37719 diff changeset	710 self.stream.close()
7c4058681224 Fix codecs.EncodedFile which did not use file_encoding in 2.5.0, and Georg Brandl <georg@python.org> parents: 37719 diff changeset	711
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	712 ###
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	713
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	714 class StreamRecoder:
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	715
12389 e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	716 """ StreamRecoder instances provide a frontend - backend
e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	717 view of encoding data.
e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	718
e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	719 They use the complete set of APIs returned by the
e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	720 codecs.lookup() function to implement their task.
e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	721
e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	722 Data written to the stream is first decoded into an
e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	723 intermediate format (which is dependent on the given codec
e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	724 combination) and then written to the stream using an instance
e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	725 of the provided Writer class.
e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	726
e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	727 In the other direction, data is read from the stream using a
e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	728 Reader instance and then return encoded data to the caller.
e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	729
e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	730 """
12367 677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	731 # Optional attributes set by the file wrappers below
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	732 data_encoding = 'unknown'
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	733 file_encoding = 'unknown'
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	734
17839 c872fdf10130 Just changed "x,y" to "x, y" everywhere (i.e., inserted horizontal space Tim Peters <tim.peters@gmail.com> parents: 16070 diff changeset	735 def __init__(self, stream, encode, decode, Reader, Writer,
c872fdf10130 Just changed "x,y" to "x, y" everywhere (i.e., inserted horizontal space Tim Peters <tim.peters@gmail.com> parents: 16070 diff changeset	736 errors='strict'):
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	737
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	738 """ Creates a StreamRecoder instance which implements a two-way
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	739 conversion: encode and decode work on the frontend (the
12373 d63b57ac617d Deleted trailing whitespace. This is really a way to be able to add Guido van Rossum <guido@python.org> parents: 12367 diff changeset	740 input to .read() and output of .write()) while
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	741 Reader and Writer work on the backend (reading and
12011 4fa25a292273 Oops, another in the same file; I should read the mail fully before Fred Drake <fdrake@acm.org> parents: 12010 diff changeset	742 writing to the stream).
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	743
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	744 You can use these objects to do transparent direct
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	745 recodings from e.g. latin-1 to utf-8 and back.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	746
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	747 stream must be a file-like object.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	748
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	749 encode, decode must adhere to the Codec interface, Reader,
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	750 Writer must be factory functions or classes providing the
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	751 StreamReader, StreamWriter interface resp.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	752
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	753 encode and decode are needed for the frontend translation,
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	754 Reader and Writer for the backend translation. Unicode is
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	755 used as intermediate encoding.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	756
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	757 Error handling is done in the same way as defined for the
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	758 StreamWriter/Readers.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	759
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	760 """
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	761 self.stream = stream
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	762 self.encode = encode
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	763 self.decode = decode
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	764 self.reader = Reader(stream, errors)
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	765 self.writer = Writer(stream, errors)
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	766 self.errors = errors
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	767
17839 c872fdf10130 Just changed "x,y" to "x, y" everywhere (i.e., inserted horizontal space Tim Peters <tim.peters@gmail.com> parents: 16070 diff changeset	768 def read(self, size=-1):
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	769
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	770 data = self.reader.read(size)
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	771 data, bytesencoded = self.encode(data, self.errors)
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	772 return data
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	773
17839 c872fdf10130 Just changed "x,y" to "x, y" everywhere (i.e., inserted horizontal space Tim Peters <tim.peters@gmail.com> parents: 16070 diff changeset	774 def readline(self, size=None):
12367 677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	775
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	776 if size is None:
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	777 data = self.reader.readline()
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	778 else:
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	779 data = self.reader.readline(size)
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	780 data, bytesencoded = self.encode(data, self.errors)
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	781 return data
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	782
17839 c872fdf10130 Just changed "x,y" to "x, y" everywhere (i.e., inserted horizontal space Tim Peters <tim.peters@gmail.com> parents: 16070 diff changeset	783 def readlines(self, sizehint=None):
12367 677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	784
31591 26a80f0af9f3 Ignore sizehint argument. Fixes SF #844561. Marc-André Lemburg <mal@egenix.com> parents: 27618 diff changeset	785 data = self.reader.read()
12367 677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	786 data, bytesencoded = self.encode(data, self.errors)
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	787 return data.splitlines(1)
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	788
26082 f3a64035ba8b Add next() and __iter__() methods to StreamReader, StreamReaderWriter Walter Dörwald <walter@livinglogic.de> parents: 25091 diff changeset	789 def next(self):
f3a64035ba8b Add next() and __iter__() methods to StreamReader, StreamReaderWriter Walter Dörwald <walter@livinglogic.de> parents: 25091 diff changeset	790
f3a64035ba8b Add next() and __iter__() methods to StreamReader, StreamReaderWriter Walter Dörwald <walter@livinglogic.de> parents: 25091 diff changeset	791 """ Return the next decoded line from the input stream."""
35508 1f2af259968b SF bug #1235646: codecs.StreamRecoder.next() now reencodes the data it reads Walter Dörwald <walter@livinglogic.de> parents: 35415 diff changeset	792 data = self.reader.next()
1f2af259968b SF bug #1235646: codecs.StreamRecoder.next() now reencodes the data it reads Walter Dörwald <walter@livinglogic.de> parents: 35415 diff changeset	793 data, bytesencoded = self.encode(data, self.errors)
1f2af259968b SF bug #1235646: codecs.StreamRecoder.next() now reencodes the data it reads Walter Dörwald <walter@livinglogic.de> parents: 35415 diff changeset	794 return data
26082 f3a64035ba8b Add next() and __iter__() methods to StreamReader, StreamReaderWriter Walter Dörwald <walter@livinglogic.de> parents: 25091 diff changeset	795
f3a64035ba8b Add next() and __iter__() methods to StreamReader, StreamReaderWriter Walter Dörwald <walter@livinglogic.de> parents: 25091 diff changeset	796 def __iter__(self):
f3a64035ba8b Add next() and __iter__() methods to StreamReader, StreamReaderWriter Walter Dörwald <walter@livinglogic.de> parents: 25091 diff changeset	797 return self
f3a64035ba8b Add next() and __iter__() methods to StreamReader, StreamReaderWriter Walter Dörwald <walter@livinglogic.de> parents: 25091 diff changeset	798
17839 c872fdf10130 Just changed "x,y" to "x, y" everywhere (i.e., inserted horizontal space Tim Peters <tim.peters@gmail.com> parents: 16070 diff changeset	799 def write(self, data):
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	800
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	801 data, bytesdecoded = self.decode(data, self.errors)
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	802 return self.writer.write(data)
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	803
17839 c872fdf10130 Just changed "x,y" to "x, y" everywhere (i.e., inserted horizontal space Tim Peters <tim.peters@gmail.com> parents: 16070 diff changeset	804 def writelines(self, list):
12367 677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	805
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	806 data = ''.join(list)
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	807 data, bytesdecoded = self.decode(data, self.errors)
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	808 return self.writer.write(data)
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	809
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	810 def reset(self):
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	811
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	812 self.reader.reset()
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	813 self.writer.reset()
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	814
17839 c872fdf10130 Just changed "x,y" to "x, y" everywhere (i.e., inserted horizontal space Tim Peters <tim.peters@gmail.com> parents: 16070 diff changeset	815 def __getattr__(self, name,
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	816 getattr=getattr):
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	817
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	818 """ Inherit all other methods from the underlying stream.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	819 """
17839 c872fdf10130 Just changed "x,y" to "x, y" everywhere (i.e., inserted horizontal space Tim Peters <tim.peters@gmail.com> parents: 16070 diff changeset	820 return getattr(self.stream, name)
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	821
40239 7c4058681224 Fix codecs.EncodedFile which did not use file_encoding in 2.5.0, and Georg Brandl <georg@python.org> parents: 37719 diff changeset	822 def __enter__(self):
7c4058681224 Fix codecs.EncodedFile which did not use file_encoding in 2.5.0, and Georg Brandl <georg@python.org> parents: 37719 diff changeset	823 return self
7c4058681224 Fix codecs.EncodedFile which did not use file_encoding in 2.5.0, and Georg Brandl <georg@python.org> parents: 37719 diff changeset	824
7c4058681224 Fix codecs.EncodedFile which did not use file_encoding in 2.5.0, and Georg Brandl <georg@python.org> parents: 37719 diff changeset	825 def __exit__(self, type, value, tb):
7c4058681224 Fix codecs.EncodedFile which did not use file_encoding in 2.5.0, and Georg Brandl <georg@python.org> parents: 37719 diff changeset	826 self.stream.close()
7c4058681224 Fix codecs.EncodedFile which did not use file_encoding in 2.5.0, and Georg Brandl <georg@python.org> parents: 37719 diff changeset	827
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	828 ### Shortcuts
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	829
12986 5279a6e00d36 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12534 diff changeset	830 def open(filename, mode='rb', encoding=None, errors='strict', buffering=1):
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	831
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	832 """ Open an encoded file using the given mode and return
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	833 a wrapped version providing transparent encoding/decoding.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	834
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	835 Note: The wrapped version will only accept the object format
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	836 defined by the codecs, i.e. Unicode objects for most builtin
34761 07745166f571 typo Skip Montanaro <skip@pobox.com> parents: 34753 diff changeset	837 codecs. Output is also codec dependent and will usually be
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	838 Unicode as well.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	839
12986 5279a6e00d36 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12534 diff changeset	840 Files are always opened in binary mode, even if no binary mode
27618 74d9604e9353 Fix typos. Walter Dörwald <walter@livinglogic.de> parents: 26906 diff changeset	841 was specified. This is done to avoid data loss due to encodings
12986 5279a6e00d36 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12534 diff changeset	842 using 8-bit values. The default file mode is 'rb' meaning to
5279a6e00d36 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12534 diff changeset	843 open the file in binary read mode.
5279a6e00d36 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12534 diff changeset	844
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	845 encoding specifies the encoding which is to be used for the
27618 74d9604e9353 Fix typos. Walter Dörwald <walter@livinglogic.de> parents: 26906 diff changeset	846 file.
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	847
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	848 errors may be given to define the error handling. It defaults
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	849 to 'strict' which causes ValueErrors to be raised in case an
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	850 encoding error occurs.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	851
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	852 buffering has the same meaning as for the builtin open() API.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	853 It defaults to line buffered.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	854
12389 e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	855 The returned wrapped file object provides an extra attribute
e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	856 .encoding which allows querying the used encoding. This
e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	857 attribute is only available if an encoding was specified as
e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	858 parameter.
e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	859
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	860 """
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	861 if encoding is not None and \
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	862 'b' not in mode:
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	863 # Force opening of the file in binary mode
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	864 mode = mode + 'b'
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	865 file = __builtin__.open(filename, mode, buffering)
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	866 if encoding is None:
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	867 return file
36996 7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	868 info = lookup(encoding)
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	869 srw = StreamReaderWriter(file, info.streamreader, info.streamwriter, errors)
12367 677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	870 # Add attributes to simplify introspection
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	871 srw.encoding = encoding
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	872 return srw
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	873
12367 677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	874 def EncodedFile(file, data_encoding, file_encoding=None, errors='strict'):
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	875
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	876 """ Return a wrapped version of file which provides transparent
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	877 encoding translation.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	878
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	879 Strings written to the wrapped file are interpreted according
12367 677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	880 to the given data_encoding and then written to the original
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	881 file as string using file_encoding. The intermediate encoding
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	882 will usually be Unicode but depends on the specified codecs.
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	883
12367 677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	884 Strings are read from the file using file_encoding and then
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	885 passed back to the caller as string using data_encoding.
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	886
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	887 If file_encoding is not given, it defaults to data_encoding.
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	888
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	889 errors may be given to define the error handling. It defaults
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	890 to 'strict' which causes ValueErrors to be raised in case an
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	891 encoding error occurs.
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	892
12389 e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	893 The returned wrapped file object provides two extra attributes
e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	894 .data_encoding and .file_encoding which reflect the given
e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	895 parameters of the same name. The attributes can be used for
e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	896 introspection by Python programs.
e5b0bea03cfa M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12373 diff changeset	897
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	898 """
12367 677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	899 if file_encoding is None:
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	900 file_encoding = data_encoding
40239 7c4058681224 Fix codecs.EncodedFile which did not use file_encoding in 2.5.0, and Georg Brandl <georg@python.org> parents: 37719 diff changeset	901 data_info = lookup(data_encoding)
7c4058681224 Fix codecs.EncodedFile which did not use file_encoding in 2.5.0, and Georg Brandl <georg@python.org> parents: 37719 diff changeset	902 file_info = lookup(file_encoding)
7c4058681224 Fix codecs.EncodedFile which did not use file_encoding in 2.5.0, and Georg Brandl <georg@python.org> parents: 37719 diff changeset	903 sr = StreamRecoder(file, data_info.encode, data_info.decode,
7c4058681224 Fix codecs.EncodedFile which did not use file_encoding in 2.5.0, and Georg Brandl <georg@python.org> parents: 37719 diff changeset	904 file_info.streamreader, file_info.streamwriter, errors)
12367 677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	905 # Add attributes to simplify introspection
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	906 sr.data_encoding = data_encoding
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	907 sr.file_encoding = file_encoding
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	908 return sr
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	909
19981 0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	910 ### Helpers for codec lookup
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	911
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	912 def getencoder(encoding):
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	913
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	914 """ Lookup up the codec for the given encoding and return
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	915 its encoder function.
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	916
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	917 Raises a LookupError in case the encoding cannot be found.
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	918
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	919 """
36996 7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	920 return lookup(encoding).encode
19981 0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	921
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	922 def getdecoder(encoding):
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	923
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	924 """ Lookup up the codec for the given encoding and return
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	925 its decoder function.
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	926
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	927 Raises a LookupError in case the encoding cannot be found.
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	928
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	929 """
36996 7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	930 return lookup(encoding).decode
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	931
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	932 def getincrementalencoder(encoding):
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	933
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	934 """ Lookup up the codec for the given encoding and return
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	935 its IncrementalEncoder class or factory function.
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	936
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	937 Raises a LookupError in case the encoding cannot be found
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	938 or the codecs doesn't provide an incremental encoder.
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	939
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	940 """
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	941 encoder = lookup(encoding).incrementalencoder
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	942 if encoder is None:
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	943 raise LookupError(encoding)
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	944 return encoder
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	945
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	946 def getincrementaldecoder(encoding):
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	947
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	948 """ Lookup up the codec for the given encoding and return
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	949 its IncrementalDecoder class or factory function.
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	950
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	951 Raises a LookupError in case the encoding cannot be found
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	952 or the codecs doesn't provide an incremental decoder.
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	953
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	954 """
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	955 decoder = lookup(encoding).incrementaldecoder
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	956 if decoder is None:
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	957 raise LookupError(encoding)
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	958 return decoder
19981 0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	959
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	960 def getreader(encoding):
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	961
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	962 """ Lookup up the codec for the given encoding and return
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	963 its StreamReader class or factory function.
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	964
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	965 Raises a LookupError in case the encoding cannot be found.
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	966
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	967 """
36996 7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	968 return lookup(encoding).streamreader
19981 0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	969
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	970 def getwriter(encoding):
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	971
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	972 """ Lookup up the codec for the given encoding and return
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	973 its StreamWriter class or factory function.
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	974
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	975 Raises a LookupError in case the encoding cannot be found.
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	976
0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	977 """
36996 7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	978 return lookup(encoding).streamwriter
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	979
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	980 def iterencode(iterator, encoding, errors='strict', **kwargs):
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	981 """
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	982 Encoding iterator.
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	983
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	984 Encodes the input strings from the iterator using a IncrementalEncoder.
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	985
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	986 errors and kwargs are passed through to the IncrementalEncoder
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	987 constructor.
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	988 """
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	989 encoder = getincrementalencoder(encoding)(errors, **kwargs)
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	990 for input in iterator:
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	991 output = encoder.encode(input)
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	992 if output:
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	993 yield output
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	994 output = encoder.encode("", True)
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	995 if output:
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	996 yield output
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	997
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	998 def iterdecode(iterator, encoding, errors='strict', **kwargs):
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	999 """
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	1000 Decoding iterator.
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	1001
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	1002 Decodes the input strings from the iterator using a IncrementalDecoder.
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	1003
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	1004 errors and kwargs are passed through to the IncrementalDecoder
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	1005 constructor.
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	1006 """
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	1007 decoder = getincrementaldecoder(encoding)(errors, **kwargs)
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	1008 for input in iterator:
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	1009 output = decoder.decode(input)
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	1010 if output:
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	1011 yield output
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	1012 output = decoder.decode("", True)
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	1013 if output:
7fa4c29d7030 Patch #1436130: codecs.lookup() now returns a CodecInfo object (a subclass Walter Dörwald <walter@livinglogic.de> parents: 36855 diff changeset	1014 yield output
19981 0fd95200ab6e Added new helpers for easy access to codecs. Docs will follow. Marc-André Lemburg <mal@egenix.com> parents: 19967 diff changeset	1015
15706 bf9bb0e3dd34 This patch changes the default behaviour of the builtin charmap Marc-André Lemburg <mal@egenix.com> parents: 15498 diff changeset	1016 ### Helpers for charmap-based codecs
bf9bb0e3dd34 This patch changes the default behaviour of the builtin charmap Marc-André Lemburg <mal@egenix.com> parents: 15498 diff changeset	1017
bf9bb0e3dd34 This patch changes the default behaviour of the builtin charmap Marc-André Lemburg <mal@egenix.com> parents: 15498 diff changeset	1018 def make_identity_dict(rng):
bf9bb0e3dd34 This patch changes the default behaviour of the builtin charmap Marc-André Lemburg <mal@egenix.com> parents: 15498 diff changeset	1019
bf9bb0e3dd34 This patch changes the default behaviour of the builtin charmap Marc-André Lemburg <mal@egenix.com> parents: 15498 diff changeset	1020 """ make_identity_dict(rng) -> dict
bf9bb0e3dd34 This patch changes the default behaviour of the builtin charmap Marc-André Lemburg <mal@egenix.com> parents: 15498 diff changeset	1021
bf9bb0e3dd34 This patch changes the default behaviour of the builtin charmap Marc-André Lemburg <mal@egenix.com> parents: 15498 diff changeset	1022 Return a dictionary where elements of the rng sequence are
bf9bb0e3dd34 This patch changes the default behaviour of the builtin charmap Marc-André Lemburg <mal@egenix.com> parents: 15498 diff changeset	1023 mapped to themselves.
15867 f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15706 diff changeset	1024
15706 bf9bb0e3dd34 This patch changes the default behaviour of the builtin charmap Marc-André Lemburg <mal@egenix.com> parents: 15498 diff changeset	1025 """
bf9bb0e3dd34 This patch changes the default behaviour of the builtin charmap Marc-André Lemburg <mal@egenix.com> parents: 15498 diff changeset	1026 res = {}
bf9bb0e3dd34 This patch changes the default behaviour of the builtin charmap Marc-André Lemburg <mal@egenix.com> parents: 15498 diff changeset	1027 for i in rng:
bf9bb0e3dd34 This patch changes the default behaviour of the builtin charmap Marc-André Lemburg <mal@egenix.com> parents: 15498 diff changeset	1028 res[i]=i
bf9bb0e3dd34 This patch changes the default behaviour of the builtin charmap Marc-André Lemburg <mal@egenix.com> parents: 15498 diff changeset	1029 return res
bf9bb0e3dd34 This patch changes the default behaviour of the builtin charmap Marc-André Lemburg <mal@egenix.com> parents: 15498 diff changeset	1030
17843 3e256f9d99ad Moved the encoding map building logic from the individual mapping Marc-André Lemburg <mal@egenix.com> parents: 17839 diff changeset	1031 def make_encoding_map(decoding_map):
3e256f9d99ad Moved the encoding map building logic from the individual mapping Marc-André Lemburg <mal@egenix.com> parents: 17839 diff changeset	1032
3e256f9d99ad Moved the encoding map building logic from the individual mapping Marc-André Lemburg <mal@egenix.com> parents: 17839 diff changeset	1033 """ Creates an encoding map from a decoding map.
3e256f9d99ad Moved the encoding map building logic from the individual mapping Marc-André Lemburg <mal@egenix.com> parents: 17839 diff changeset	1034
27618 74d9604e9353 Fix typos. Walter Dörwald <walter@livinglogic.de> parents: 26906 diff changeset	1035 If a target mapping in the decoding map occurs multiple
17843 3e256f9d99ad Moved the encoding map building logic from the individual mapping Marc-André Lemburg <mal@egenix.com> parents: 17839 diff changeset	1036 times, then that target is mapped to None (undefined mapping),
3e256f9d99ad Moved the encoding map building logic from the individual mapping Marc-André Lemburg <mal@egenix.com> parents: 17839 diff changeset	1037 causing an exception when encountered by the charmap codec
3e256f9d99ad Moved the encoding map building logic from the individual mapping Marc-André Lemburg <mal@egenix.com> parents: 17839 diff changeset	1038 during translation.
3e256f9d99ad Moved the encoding map building logic from the individual mapping Marc-André Lemburg <mal@egenix.com> parents: 17839 diff changeset	1039
3e256f9d99ad Moved the encoding map building logic from the individual mapping Marc-André Lemburg <mal@egenix.com> parents: 17839 diff changeset	1040 One example where this happens is cp875.py which decodes
3e256f9d99ad Moved the encoding map building logic from the individual mapping Marc-André Lemburg <mal@egenix.com> parents: 17839 diff changeset	1041 multiple character to \u001a.
3e256f9d99ad Moved the encoding map building logic from the individual mapping Marc-André Lemburg <mal@egenix.com> parents: 17839 diff changeset	1042
3e256f9d99ad Moved the encoding map building logic from the individual mapping Marc-André Lemburg <mal@egenix.com> parents: 17839 diff changeset	1043 """
3e256f9d99ad Moved the encoding map building logic from the individual mapping Marc-André Lemburg <mal@egenix.com> parents: 17839 diff changeset	1044 m = {}
3e256f9d99ad Moved the encoding map building logic from the individual mapping Marc-André Lemburg <mal@egenix.com> parents: 17839 diff changeset	1045 for k,v in decoding_map.items():
23620 a1de0bab5475 SF 563203. Replaced 'has_key()' with 'in'. Raymond Hettinger <python@rcn.com> parents: 22149 diff changeset	1046 if not v in m:
17843 3e256f9d99ad Moved the encoding map building logic from the individual mapping Marc-André Lemburg <mal@egenix.com> parents: 17839 diff changeset	1047 m[v] = k
3e256f9d99ad Moved the encoding map building logic from the individual mapping Marc-André Lemburg <mal@egenix.com> parents: 17839 diff changeset	1048 else:
3e256f9d99ad Moved the encoding map building logic from the individual mapping Marc-André Lemburg <mal@egenix.com> parents: 17839 diff changeset	1049 m[v] = None
3e256f9d99ad Moved the encoding map building logic from the individual mapping Marc-André Lemburg <mal@egenix.com> parents: 17839 diff changeset	1050 return m
17978 5e87a08abb05 Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 17843 diff changeset	1051
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 23658 diff changeset	1052 ### error handlers
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 23658 diff changeset	1053
34707 0d44d9071e97 Build with --disable-unicode again. Fixes #1158607. Martin v. Löwis <martin@v.loewis.de> parents: 34387 diff changeset	1054 try:
0d44d9071e97 Build with --disable-unicode again. Fixes #1158607. Martin v. Löwis <martin@v.loewis.de> parents: 34387 diff changeset	1055 strict_errors = lookup_error("strict")
0d44d9071e97 Build with --disable-unicode again. Fixes #1158607. Martin v. Löwis <martin@v.loewis.de> parents: 34387 diff changeset	1056 ignore_errors = lookup_error("ignore")
0d44d9071e97 Build with --disable-unicode again. Fixes #1158607. Martin v. Löwis <martin@v.loewis.de> parents: 34387 diff changeset	1057 replace_errors = lookup_error("replace")
0d44d9071e97 Build with --disable-unicode again. Fixes #1158607. Martin v. Löwis <martin@v.loewis.de> parents: 34387 diff changeset	1058 xmlcharrefreplace_errors = lookup_error("xmlcharrefreplace")
0d44d9071e97 Build with --disable-unicode again. Fixes #1158607. Martin v. Löwis <martin@v.loewis.de> parents: 34387 diff changeset	1059 backslashreplace_errors = lookup_error("backslashreplace")
0d44d9071e97 Build with --disable-unicode again. Fixes #1158607. Martin v. Löwis <martin@v.loewis.de> parents: 34387 diff changeset	1060 except LookupError:
0d44d9071e97 Build with --disable-unicode again. Fixes #1158607. Martin v. Löwis <martin@v.loewis.de> parents: 34387 diff changeset	1061 # In --disable-unicode builds, these error handler are missing
0d44d9071e97 Build with --disable-unicode again. Fixes #1158607. Martin v. Löwis <martin@v.loewis.de> parents: 34387 diff changeset	1062 strict_errors = None
0d44d9071e97 Build with --disable-unicode again. Fixes #1158607. Martin v. Löwis <martin@v.loewis.de> parents: 34387 diff changeset	1063 ignore_errors = None
0d44d9071e97 Build with --disable-unicode again. Fixes #1158607. Martin v. Löwis <martin@v.loewis.de> parents: 34387 diff changeset	1064 replace_errors = None
0d44d9071e97 Build with --disable-unicode again. Fixes #1158607. Martin v. Löwis <martin@v.loewis.de> parents: 34387 diff changeset	1065 xmlcharrefreplace_errors = None
0d44d9071e97 Build with --disable-unicode again. Fixes #1158607. Martin v. Löwis <martin@v.loewis.de> parents: 34387 diff changeset	1066 backslashreplace_errors = None
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 23658 diff changeset	1067
18904 5af1ee422575 Add dead imports of modules that are "magically" imported. Martin v. Löwis <martin@v.loewis.de> parents: 17978 diff changeset	1068 # Tell modulefinder that using codecs probably needs the encodings
5af1ee422575 Add dead imports of modules that are "magically" imported. Martin v. Löwis <martin@v.loewis.de> parents: 17978 diff changeset	1069 # package
5af1ee422575 Add dead imports of modules that are "magically" imported. Martin v. Löwis <martin@v.loewis.de> parents: 17978 diff changeset	1070 _false = 0
5af1ee422575 Add dead imports of modules that are "magically" imported. Martin v. Löwis <martin@v.loewis.de> parents: 17978 diff changeset	1071 if _false:
5af1ee422575 Add dead imports of modules that are "magically" imported. Martin v. Löwis <martin@v.loewis.de> parents: 17978 diff changeset	1072 import encodings
5af1ee422575 Add dead imports of modules that are "magically" imported. Martin v. Löwis <martin@v.loewis.de> parents: 17978 diff changeset	1073
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	1074 ### Tests
12373 d63b57ac617d Deleted trailing whitespace. This is really a way to be able to add Guido van Rossum <guido@python.org> parents: 12367 diff changeset	1075
11990 f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	1076 if __name__ == '__main__':
f104251b6f8f Module codecs -- Python Codec Registry, API and helpers. Written by Guido van Rossum <guido@python.org> parents: diff changeset	1077
12367 677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	1078 # Make stdout translate Latin-1 output into UTF-8 output
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	1079 sys.stdout = EncodedFile(sys.stdout, 'latin-1', 'utf-8')
12373 d63b57ac617d Deleted trailing whitespace. This is really a way to be able to add Guido van Rossum <guido@python.org> parents: 12367 diff changeset	1080
12367 677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	1081 # Have stdin translate Latin-1 input into UTF-8 input
677e2147829a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12195 diff changeset	1082 sys.stdin = EncodedFile(sys.stdin, 'utf-8', 'latin-1')

Mercurial > cpython-lru_cache-weakref

annotate Lib/codecs.py @ 58765:2ba7677bfa95 legacy-trunk