cpython-lru_cache-weakref: Lib/email/header.py annotate

annotate Lib/email/header.py @ 58925:eecf1bef052d 2.6

Merged revisions 78207 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r78207 | ezio.melotti | 2010-02-17 01:26:09 +0200 (Wed, 17 Feb 2010) | 1 line #7930: fix stripid ........

author	Ezio Melotti <ezio.melotti@gmail.com>
date	Tue, 16 Feb 2010 23:29:44 +0000
parents	a1b79778f22e
children	fca3c9fb5a1b

rev	line source
37078 3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	1 # Copyright (C) 2002-2006 Python Software Foundation
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	2 # Author: Ben Gertzfield, Barry Warsaw
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	3 # Contact: email-sig@python.org
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	4
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	5 """Header encoding and decoding functionality."""
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	6
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	7 __all__ = [
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	8 'Header',
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	9 'decode_header',
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	10 'make_header',
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	11 ]
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	12
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	13 import re
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	14 import binascii
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	15
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	16 import email.quoprimime
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	17 import email.base64mime
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	18
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	19 from email.errors import HeaderParseError
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	20 from email.charset import Charset
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	21
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	22 NL = '\n'
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	23 SPACE = ' '
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	24 USPACE = u' '
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	25 SPACE8 = ' ' * 8
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	26 UEMPTYSTRING = u''
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	27
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	28 MAXLINELEN = 76
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	29
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	30 USASCII = Charset('us-ascii')
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	31 UTF8 = Charset('utf-8')
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	32
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	33 # Match encoded-word strings in the form =?charset?q?Hello_World?=
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	34 ecre = re.compile(r'''
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	35 =\? # literal =?
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	36 (?P<charset>[^?]*?) # non-greedy up to the next ? is the charset
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	37 \? # literal ?
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	38 (?P<encoding>[qb]) # either a "q" or a "b", case insensitive
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	39 \? # literal ?
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	40 (?P<encoded>.*?) # non-greedy up to the next ?= is the encoded string
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	41 \?= # literal ?=
41181 de1d05c55b6f SF bug #1582282; decode_header() incorrectly splits not-conformant RFC Barry Warsaw <barry@python.org> parents: 37078 diff changeset	42 (?=[ \t]\|$) # whitespace or the end of the string
de1d05c55b6f SF bug #1582282; decode_header() incorrectly splits not-conformant RFC Barry Warsaw <barry@python.org> parents: 37078 diff changeset	43 ''', re.VERBOSE \| re.IGNORECASE \| re.MULTILINE)
37078 3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	44
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	45 # Field name regexp, including trailing colon, but not separating whitespace,
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	46 # according to RFC 2822. Character range is from tilde to exclamation mark.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	47 # For use with .match()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	48 fcre = re.compile(r'[\041-\176]+:$')
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	49
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	50
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	51
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	52 # Helpers
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	53 _max_append = email.quoprimime._max_append
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	54
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	55
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	56
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	57 def decode_header(header):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	58 """Decode a message header value without converting charset.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	59
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	60 Returns a list of (decoded_string, charset) pairs containing each of the
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	61 decoded parts of the header. Charset is None for non-encoded parts of the
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	62 header, otherwise a lower-case string containing the name of the character
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	63 set specified in the encoded string.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	64
55215 a1b79778f22e Merged revisions 73952 via svnmerge from Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 48149 diff changeset	65 An email.errors.HeaderParseError may be raised when certain decoding error
37078 3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	66 occurs (e.g. a base64 decoding exception).
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	67 """
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	68 # If no encoding, just return the header
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	69 header = str(header)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	70 if not ecre.search(header):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	71 return [(header, None)]
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	72 decoded = []
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	73 dec = ''
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	74 for line in header.splitlines():
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	75 # This line might not have an encoding in it
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	76 if not ecre.search(line):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	77 decoded.append((line, None))
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	78 continue
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	79 parts = ecre.split(line)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	80 while parts:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	81 unenc = parts.pop(0).strip()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	82 if unenc:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	83 # Should we continue a long line?
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	84 if decoded and decoded[-1][1] is None:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	85 decoded[-1] = (decoded[-1][0] + SPACE + unenc, None)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	86 else:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	87 decoded.append((unenc, None))
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	88 if parts:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	89 charset, encoding = [s.lower() for s in parts[0:2]]
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	90 encoded = parts[2]
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	91 dec = None
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	92 if encoding == 'q':
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	93 dec = email.quoprimime.header_decode(encoded)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	94 elif encoding == 'b':
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	95 try:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	96 dec = email.base64mime.decode(encoded)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	97 except binascii.Error:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	98 # Turn this into a higher level exception. BAW: Right
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	99 # now we throw the lower level exception away but
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	100 # when/if we get exception chaining, we'll preserve it.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	101 raise HeaderParseError
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	102 if dec is None:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	103 dec = encoded
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	104
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	105 if decoded and decoded[-1][1] == charset:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	106 decoded[-1] = (decoded[-1][0] + dec, decoded[-1][1])
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	107 else:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	108 decoded.append((dec, charset))
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	109 del parts[0:3]
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	110 return decoded
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	111
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	112
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	113
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	114 def make_header(decoded_seq, maxlinelen=None, header_name=None,
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	115 continuation_ws=' '):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	116 """Create a Header from a sequence of pairs as returned by decode_header()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	117
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	118 decode_header() takes a header value string and returns a sequence of
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	119 pairs of the format (decoded_string, charset) where charset is the string
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	120 name of the character set.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	121
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	122 This function takes one of those sequence of pairs and returns a Header
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	123 instance. Optional maxlinelen, header_name, and continuation_ws are as in
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	124 the Header constructor.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	125 """
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	126 h = Header(maxlinelen=maxlinelen, header_name=header_name,
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	127 continuation_ws=continuation_ws)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	128 for s, charset in decoded_seq:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	129 # None means us-ascii but we can simply pass it on to h.append()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	130 if charset is not None and not isinstance(charset, Charset):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	131 charset = Charset(charset)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	132 h.append(s, charset)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	133 return h
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	134
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	135
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	136
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	137 class Header:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	138 def __init__(self, s=None, charset=None,
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	139 maxlinelen=None, header_name=None,
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	140 continuation_ws=' ', errors='strict'):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	141 """Create a MIME-compliant header that can contain many character sets.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	142
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	143 Optional s is the initial header value. If None, the initial header
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	144 value is not set. You can later append to the header with .append()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	145 method calls. s may be a byte string or a Unicode string, but see the
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	146 .append() documentation for semantics.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	147
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	148 Optional charset serves two purposes: it has the same meaning as the
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	149 charset argument to the .append() method. It also sets the default
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	150 character set for all subsequent .append() calls that omit the charset
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	151 argument. If charset is not provided in the constructor, the us-ascii
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	152 charset is used both as s's initial charset and as the default for
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	153 subsequent .append() calls.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	154
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	155 The maximum line length can be specified explicit via maxlinelen. For
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	156 splitting the first line to a shorter value (to account for the field
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	157 header which isn't included in s, e.g. `Subject') pass in the name of
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	158 the field in header_name. The default maxlinelen is 76.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	159
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	160 continuation_ws must be RFC 2822 compliant folding whitespace (usually
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	161 either a space or a hard tab) which will be prepended to continuation
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	162 lines.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	163
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	164 errors is passed through to the .append() call.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	165 """
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	166 if charset is None:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	167 charset = USASCII
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	168 if not isinstance(charset, Charset):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	169 charset = Charset(charset)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	170 self._charset = charset
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	171 self._continuation_ws = continuation_ws
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	172 cws_expanded_len = len(continuation_ws.replace('\t', SPACE8))
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	173 # BAW: I believe `chunks' and `maxlinelen' should be non-public.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	174 self._chunks = []
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	175 if s is not None:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	176 self.append(s, charset, errors)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	177 if maxlinelen is None:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	178 maxlinelen = MAXLINELEN
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	179 if header_name is None:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	180 # We don't know anything about the field header so the first line
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	181 # is the same length as subsequent lines.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	182 self._firstlinelen = maxlinelen
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	183 else:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	184 # The first line should be shorter to take into account the field
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	185 # header. Also subtract off 2 extra for the colon and space.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	186 self._firstlinelen = maxlinelen - len(header_name) - 2
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	187 # Second and subsequent lines should subtract off the length in
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	188 # columns of the continuation whitespace prefix.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	189 self._maxlinelen = maxlinelen - cws_expanded_len
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	190
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	191 def __str__(self):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	192 """A synonym for self.encode()."""
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	193 return self.encode()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	194
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	195 def __unicode__(self):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	196 """Helper for the built-in unicode function."""
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	197 uchunks = []
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	198 lastcs = None
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	199 for s, charset in self._chunks:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	200 # We must preserve spaces between encoded and non-encoded word
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	201 # boundaries, which means for us we need to add a space when we go
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	202 # from a charset to None/us-ascii, or from None/us-ascii to a
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	203 # charset. Only do this for the second and subsequent chunks.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	204 nextcs = charset
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	205 if uchunks:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	206 if lastcs not in (None, 'us-ascii'):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	207 if nextcs in (None, 'us-ascii'):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	208 uchunks.append(USPACE)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	209 nextcs = None
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	210 elif nextcs not in (None, 'us-ascii'):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	211 uchunks.append(USPACE)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	212 lastcs = nextcs
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	213 uchunks.append(unicode(s, str(charset)))
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	214 return UEMPTYSTRING.join(uchunks)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	215
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	216 # Rich comparison operators for equality only. BAW: does it make sense to
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	217 # have or explicitly disable <, <=, >, >= operators?
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	218 def __eq__(self, other):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	219 # other may be a Header or a string. Both are fine so coerce
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	220 # ourselves to a string, swap the args and do another comparison.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	221 return other == self.encode()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	222
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	223 def __ne__(self, other):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	224 return not self == other
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	225
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	226 def append(self, s, charset=None, errors='strict'):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	227 """Append a string to the MIME header.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	228
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	229 Optional charset, if given, should be a Charset instance or the name
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	230 of a character set (which will be converted to a Charset instance). A
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	231 value of None (the default) means that the charset given in the
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	232 constructor is used.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	233
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	234 s may be a byte string or a Unicode string. If it is a byte string
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	235 (i.e. isinstance(s, str) is true), then charset is the encoding of
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	236 that byte string, and a UnicodeError will be raised if the string
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	237 cannot be decoded with that charset. If s is a Unicode string, then
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	238 charset is a hint specifying the character set of the characters in
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	239 the string. In this case, when producing an RFC 2822 compliant header
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	240 using RFC 2047 rules, the Unicode string will be encoded using the
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	241 following charsets in order: us-ascii, the charset hint, utf-8. The
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	242 first character set not to provoke a UnicodeError is used.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	243
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	244 Optional `errors' is passed as the third argument to any unicode() or
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	245 ustr.encode() call.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	246 """
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	247 if charset is None:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	248 charset = self._charset
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	249 elif not isinstance(charset, Charset):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	250 charset = Charset(charset)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	251 # If the charset is our faux 8bit charset, leave the string unchanged
48149 1f22de93daf8 Remove Barry's love of deprecated syntax to silence warnings in the email Brett Cannon <bcannon@gmail.com> parents: 41181 diff changeset	252 if charset != '8bit':
37078 3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	253 # We need to test that the string can be converted to unicode and
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	254 # back to a byte string, given the input and output codecs of the
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	255 # charset.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	256 if isinstance(s, str):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	257 # Possibly raise UnicodeError if the byte string can't be
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	258 # converted to a unicode with the input codec of the charset.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	259 incodec = charset.input_codec or 'us-ascii'
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	260 ustr = unicode(s, incodec, errors)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	261 # Now make sure that the unicode could be converted back to a
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	262 # byte string with the output codec, which may be different
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	263 # than the iput coded. Still, use the original byte string.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	264 outcodec = charset.output_codec or 'us-ascii'
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	265 ustr.encode(outcodec, errors)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	266 elif isinstance(s, unicode):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	267 # Now we have to be sure the unicode string can be converted
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	268 # to a byte string with a reasonable output codec. We want to
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	269 # use the byte string in the chunk.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	270 for charset in USASCII, charset, UTF8:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	271 try:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	272 outcodec = charset.output_codec or 'us-ascii'
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	273 s = s.encode(outcodec, errors)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	274 break
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	275 except UnicodeError:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	276 pass
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	277 else:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	278 assert False, 'utf-8 conversion failed'
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	279 self._chunks.append((s, charset))
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	280
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	281 def _split(self, s, charset, maxlinelen, splitchars):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	282 # Split up a header safely for use with encode_chunks.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	283 splittable = charset.to_splittable(s)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	284 encoded = charset.from_splittable(splittable, True)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	285 elen = charset.encoded_header_len(encoded)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	286 # If the line's encoded length first, just return it
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	287 if elen <= maxlinelen:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	288 return [(encoded, charset)]
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	289 # If we have undetermined raw 8bit characters sitting in a byte
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	290 # string, we really don't know what the right thing to do is. We
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	291 # can't really split it because it might be multibyte data which we
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	292 # could break if we split it between pairs. The least harm seems to
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	293 # be to not split the header at all, but that means they could go out
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	294 # longer than maxlinelen.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	295 if charset == '8bit':
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	296 return [(s, charset)]
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	297 # BAW: I'm not sure what the right test here is. What we're trying to
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	298 # do is be faithful to RFC 2822's recommendation that ($2.2.3):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	299 #
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	300 # "Note: Though structured field bodies are defined in such a way that
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	301 # folding can take place between many of the lexical tokens (and even
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	302 # within some of the lexical tokens), folding SHOULD be limited to
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	303 # placing the CRLF at higher-level syntactic breaks."
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	304 #
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	305 # For now, I can only imagine doing this when the charset is us-ascii,
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	306 # although it's possible that other charsets may also benefit from the
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	307 # higher-level syntactic breaks.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	308 elif charset == 'us-ascii':
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	309 return self._split_ascii(s, charset, maxlinelen, splitchars)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	310 # BAW: should we use encoded?
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	311 elif elen == len(s):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	312 # We can split on _maxlinelen boundaries because we know that the
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	313 # encoding won't change the size of the string
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	314 splitpnt = maxlinelen
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	315 first = charset.from_splittable(splittable[:splitpnt], False)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	316 last = charset.from_splittable(splittable[splitpnt:], False)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	317 else:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	318 # Binary search for split point
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	319 first, last = _binsplit(splittable, charset, maxlinelen)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	320 # first is of the proper length so just wrap it in the appropriate
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	321 # chrome. last must be recursively split.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	322 fsplittable = charset.to_splittable(first)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	323 fencoded = charset.from_splittable(fsplittable, True)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	324 chunk = [(fencoded, charset)]
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	325 return chunk + self._split(last, charset, self._maxlinelen, splitchars)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	326
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	327 def _split_ascii(self, s, charset, firstlen, splitchars):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	328 chunks = _split_ascii(s, firstlen, self._maxlinelen,
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	329 self._continuation_ws, splitchars)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	330 return zip(chunks, [charset]*len(chunks))
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	331
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	332 def _encode_chunks(self, newchunks, maxlinelen):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	333 # MIME-encode a header with many different charsets and/or encodings.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	334 #
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	335 # Given a list of pairs (string, charset), return a MIME-encoded
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	336 # string suitable for use in a header field. Each pair may have
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	337 # different charsets and/or encodings, and the resulting header will
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	338 # accurately reflect each setting.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	339 #
55215 a1b79778f22e Merged revisions 73952 via svnmerge from Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 48149 diff changeset	340 # Each encoding can be email.utils.QP (quoted-printable, for
a1b79778f22e Merged revisions 73952 via svnmerge from Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 48149 diff changeset	341 # ASCII-like character sets like iso-8859-1), email.utils.BASE64
37078 3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	342 # (Base64, for non-ASCII like character sets like KOI8-R and
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	343 # iso-2022-jp), or None (no encoding).
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	344 #
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	345 # Each pair will be represented on a separate line; the resulting
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	346 # string will be in the format:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	347 #
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	348 # =?charset1?q?Mar=EDa_Gonz=E1lez_Alonso?=\n
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	349 # =?charset2?b?SvxyZ2VuIEL2aW5n?="
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	350 chunks = []
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	351 for header, charset in newchunks:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	352 if not header:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	353 continue
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	354 if charset is None or charset.header_encoding is None:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	355 s = header
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	356 else:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	357 s = charset.header_encode(header)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	358 # Don't add more folding whitespace than necessary
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	359 if chunks and chunks[-1].endswith(' '):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	360 extra = ''
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	361 else:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	362 extra = ' '
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	363 _max_append(chunks, s, maxlinelen, extra)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	364 joiner = NL + self._continuation_ws
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	365 return joiner.join(chunks)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	366
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	367 def encode(self, splitchars=';, '):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	368 """Encode a message header into an RFC-compliant format.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	369
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	370 There are many issues involved in converting a given string for use in
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	371 an email header. Only certain character sets are readable in most
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	372 email clients, and as header strings can only contain a subset of
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	373 7-bit ASCII, care must be taken to properly convert and encode (with
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	374 Base64 or quoted-printable) header strings. In addition, there is a
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	375 75-character length limit on any given encoded header field, so
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	376 line-wrapping must be performed, even with double-byte character sets.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	377
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	378 This method will do its best to convert the string to the correct
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	379 character set used in email, and encode and line wrap it safely with
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	380 the appropriate scheme for that character set.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	381
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	382 If the given charset is not known or an error occurs during
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	383 conversion, this function will return the header untouched.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	384
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	385 Optional splitchars is a string containing characters to split long
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	386 ASCII lines on, in rough support of RFC 2822's `highest level
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	387 syntactic breaks'. This doesn't affect RFC 2047 encoded lines.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	388 """
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	389 newchunks = []
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	390 maxlinelen = self._firstlinelen
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	391 lastlen = 0
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	392 for s, charset in self._chunks:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	393 # The first bit of the next chunk should be just long enough to
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	394 # fill the next line. Don't forget the space separating the
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	395 # encoded words.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	396 targetlen = maxlinelen - lastlen - 1
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	397 if targetlen < charset.encoded_header_len(''):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	398 # Stick it on the next line
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	399 targetlen = maxlinelen
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	400 newchunks += self._split(s, charset, targetlen, splitchars)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	401 lastchunk, lastcharset = newchunks[-1]
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	402 lastlen = lastcharset.encoded_header_len(lastchunk)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	403 return self._encode_chunks(newchunks, maxlinelen)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	404
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	405
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	406
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	407 def _split_ascii(s, firstlen, restlen, continuation_ws, splitchars):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	408 lines = []
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	409 maxlen = firstlen
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	410 for line in s.splitlines():
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	411 # Ignore any leading whitespace (i.e. continuation whitespace) already
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	412 # on the line, since we'll be adding our own.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	413 line = line.lstrip()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	414 if len(line) < maxlen:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	415 lines.append(line)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	416 maxlen = restlen
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	417 continue
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	418 # Attempt to split the line at the highest-level syntactic break
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	419 # possible. Note that we don't have a lot of smarts about field
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	420 # syntax; we just try to break on semi-colons, then commas, then
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	421 # whitespace.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	422 for ch in splitchars:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	423 if ch in line:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	424 break
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	425 else:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	426 # There's nothing useful to split the line on, not even spaces, so
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	427 # just append this line unchanged
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	428 lines.append(line)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	429 maxlen = restlen
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	430 continue
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	431 # Now split the line on the character plus trailing whitespace
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	432 cre = re.compile(r'%s\s*' % ch)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	433 if ch in ';,':
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	434 eol = ch
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	435 else:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	436 eol = ''
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	437 joiner = eol + ' '
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	438 joinlen = len(joiner)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	439 wslen = len(continuation_ws.replace('\t', SPACE8))
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	440 this = []
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	441 linelen = 0
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	442 for part in cre.split(line):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	443 curlen = linelen + max(0, len(this)-1) * joinlen
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	444 partlen = len(part)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	445 onfirstline = not lines
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	446 # We don't want to split after the field name, if we're on the
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	447 # first line and the field name is present in the header string.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	448 if ch == ' ' and onfirstline and \
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	449 len(this) == 1 and fcre.match(this[0]):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	450 this.append(part)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	451 linelen += partlen
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	452 elif curlen + partlen > maxlen:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	453 if this:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	454 lines.append(joiner.join(this) + eol)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	455 # If this part is longer than maxlen and we aren't already
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	456 # splitting on whitespace, try to recursively split this line
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	457 # on whitespace.
48149 1f22de93daf8 Remove Barry's love of deprecated syntax to silence warnings in the email Brett Cannon <bcannon@gmail.com> parents: 41181 diff changeset	458 if partlen > maxlen and ch != ' ':
37078 3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	459 subl = _split_ascii(part, maxlen, restlen,
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	460 continuation_ws, ' ')
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	461 lines.extend(subl[:-1])
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	462 this = [subl[-1]]
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	463 else:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	464 this = [part]
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	465 linelen = wslen + len(this[-1])
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	466 maxlen = restlen
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	467 else:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	468 this.append(part)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	469 linelen += partlen
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	470 # Put any left over parts on a line by themselves
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	471 if this:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	472 lines.append(joiner.join(this))
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	473 return lines
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	474
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	475
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	476
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	477 def _binsplit(splittable, charset, maxlinelen):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	478 i = 0
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	479 j = len(splittable)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	480 while i < j:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	481 # Invariants:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	482 # 1. splittable[:k] fits for all k <= i (note that we assume,
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	483 # at the start, that splittable[:0] fits).
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	484 # 2. splittable[:k] does not fit for any k > j (at the start,
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	485 # this means we shouldn't look at any k > len(splittable)).
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	486 # 3. We don't know about splittable[:k] for k in i+1..j.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	487 # 4. We want to set i to the largest k that fits, with i <= k <= j.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	488 #
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	489 m = (i+j+1) >> 1 # ceiling((i+j)/2); i < m <= j
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	490 chunk = charset.from_splittable(splittable[:m], True)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	491 chunklen = charset.encoded_header_len(chunk)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	492 if chunklen <= maxlinelen:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	493 # m is acceptable, so is a new lower bound.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	494 i = m
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	495 else:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	496 # m is not acceptable, so final i must be < m.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	497 j = m - 1
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	498 # i == j. Invariant #1 implies that splittable[:i] fits, and
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	499 # invariant #2 implies that splittable[:i+1] does not fit, so i
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	500 # is what we're looking for.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	501 first = charset.from_splittable(splittable[:i], False)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	502 last = charset.from_splittable(splittable[i:], False)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	503 return first, last

Mercurial > cpython-lru_cache-weakref

annotate Lib/email/header.py @ 58925:eecf1bef052d 2.6