cpython-lru_cache-weakref: Lib/email/header.py annotate

annotate Lib/email/header.py @ 80661:94de77bd0b4b 2.7

Issue #16194: document imp.load_dynamic problems when read several extension modules from the same binary object file.

author	Andrew Svetlov <andrew.svetlov@gmail.com>
date	Thu, 29 Nov 2012 19:51:28 +0200
parents	cd8d3c2d3bce
children	8c2635afbfe1

rev	line source
37078 3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	1 # Copyright (C) 2002-2006 Python Software Foundation
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	2 # Author: Ben Gertzfield, Barry Warsaw
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	3 # Contact: email-sig@python.org
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	4
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	5 """Header encoding and decoding functionality."""
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	6
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	7 __all__ = [
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	8 'Header',
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	9 'decode_header',
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	10 'make_header',
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	11 ]
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	12
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	13 import re
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	14 import binascii
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	15
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	16 import email.quoprimime
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	17 import email.base64mime
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	18
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	19 from email.errors import HeaderParseError
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	20 from email.charset import Charset
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	21
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	22 NL = '\n'
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	23 SPACE = ' '
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	24 USPACE = u' '
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	25 SPACE8 = ' ' * 8
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	26 UEMPTYSTRING = u''
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	27
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	28 MAXLINELEN = 76
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	29
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	30 USASCII = Charset('us-ascii')
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	31 UTF8 = Charset('utf-8')
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	32
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	33 # Match encoded-word strings in the form =?charset?q?Hello_World?=
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	34 ecre = re.compile(r'''
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	35 =\? # literal =?
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	36 (?P<charset>[^?]*?) # non-greedy up to the next ? is the charset
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	37 \? # literal ?
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	38 (?P<encoding>[qb]) # either a "q" or a "b", case insensitive
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	39 \? # literal ?
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	40 (?P<encoded>.*?) # non-greedy up to the next ?= is the encoded string
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	41 \?= # literal ?=
41181 de1d05c55b6f SF bug #1582282; decode_header() incorrectly splits not-conformant RFC Barry Warsaw <barry@python.org> parents: 37078 diff changeset	42 (?=[ \t]\|$) # whitespace or the end of the string
de1d05c55b6f SF bug #1582282; decode_header() incorrectly splits not-conformant RFC Barry Warsaw <barry@python.org> parents: 37078 diff changeset	43 ''', re.VERBOSE \| re.IGNORECASE \| re.MULTILINE)
37078 3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	44
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	45 # Field name regexp, including trailing colon, but not separating whitespace,
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	46 # according to RFC 2822. Character range is from tilde to exclamation mark.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	47 # For use with .match()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	48 fcre = re.compile(r'[\041-\176]+:$')
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	49
68588 cd8d3c2d3bce #11565: Fix several typos. Patch by Piotr Kasprzyk. Ezio Melotti <ezio.melotti@gmail.com> parents: 67478 diff changeset	50 # Find a header embedded in a putative header value. Used to check for
67478 adc7e1671d8c Merged revisions 87873 via svnmerge from R. David Murray <rdmurray@bitdance.com> parents: 63709 diff changeset	51 # header injection attack.
adc7e1671d8c Merged revisions 87873 via svnmerge from R. David Murray <rdmurray@bitdance.com> parents: 63709 diff changeset	52 _embeded_header = re.compile(r'\n[^ \t]+:')
adc7e1671d8c Merged revisions 87873 via svnmerge from R. David Murray <rdmurray@bitdance.com> parents: 63709 diff changeset	53
37078 3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	54
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	55
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	56 # Helpers
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	57 _max_append = email.quoprimime._max_append
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	58
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	59
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	60
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	61 def decode_header(header):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	62 """Decode a message header value without converting charset.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	63
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	64 Returns a list of (decoded_string, charset) pairs containing each of the
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	65 decoded parts of the header. Charset is None for non-encoded parts of the
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	66 header, otherwise a lower-case string containing the name of the character
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	67 set specified in the encoded string.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	68
55215 a1b79778f22e Merged revisions 73952 via svnmerge from Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 48149 diff changeset	69 An email.errors.HeaderParseError may be raised when certain decoding error
37078 3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	70 occurs (e.g. a base64 decoding exception).
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	71 """
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	72 # If no encoding, just return the header
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	73 header = str(header)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	74 if not ecre.search(header):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	75 return [(header, None)]
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	76 decoded = []
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	77 dec = ''
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	78 for line in header.splitlines():
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	79 # This line might not have an encoding in it
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	80 if not ecre.search(line):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	81 decoded.append((line, None))
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	82 continue
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	83 parts = ecre.split(line)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	84 while parts:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	85 unenc = parts.pop(0).strip()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	86 if unenc:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	87 # Should we continue a long line?
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	88 if decoded and decoded[-1][1] is None:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	89 decoded[-1] = (decoded[-1][0] + SPACE + unenc, None)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	90 else:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	91 decoded.append((unenc, None))
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	92 if parts:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	93 charset, encoding = [s.lower() for s in parts[0:2]]
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	94 encoded = parts[2]
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	95 dec = None
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	96 if encoding == 'q':
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	97 dec = email.quoprimime.header_decode(encoded)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	98 elif encoding == 'b':
63709 fca3c9fb5a1b Merged revisions 83690 via svnmerge from R. David Murray <rdmurray@bitdance.com> parents: 55215 diff changeset	99 paderr = len(encoded) % 4 # Postel's law: add missing padding
fca3c9fb5a1b Merged revisions 83690 via svnmerge from R. David Murray <rdmurray@bitdance.com> parents: 55215 diff changeset	100 if paderr:
fca3c9fb5a1b Merged revisions 83690 via svnmerge from R. David Murray <rdmurray@bitdance.com> parents: 55215 diff changeset	101 encoded += '==='[:4 - paderr]
37078 3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	102 try:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	103 dec = email.base64mime.decode(encoded)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	104 except binascii.Error:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	105 # Turn this into a higher level exception. BAW: Right
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	106 # now we throw the lower level exception away but
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	107 # when/if we get exception chaining, we'll preserve it.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	108 raise HeaderParseError
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	109 if dec is None:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	110 dec = encoded
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	111
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	112 if decoded and decoded[-1][1] == charset:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	113 decoded[-1] = (decoded[-1][0] + dec, decoded[-1][1])
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	114 else:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	115 decoded.append((dec, charset))
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	116 del parts[0:3]
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	117 return decoded
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	118
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	119
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	120
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	121 def make_header(decoded_seq, maxlinelen=None, header_name=None,
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	122 continuation_ws=' '):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	123 """Create a Header from a sequence of pairs as returned by decode_header()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	124
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	125 decode_header() takes a header value string and returns a sequence of
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	126 pairs of the format (decoded_string, charset) where charset is the string
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	127 name of the character set.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	128
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	129 This function takes one of those sequence of pairs and returns a Header
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	130 instance. Optional maxlinelen, header_name, and continuation_ws are as in
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	131 the Header constructor.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	132 """
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	133 h = Header(maxlinelen=maxlinelen, header_name=header_name,
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	134 continuation_ws=continuation_ws)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	135 for s, charset in decoded_seq:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	136 # None means us-ascii but we can simply pass it on to h.append()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	137 if charset is not None and not isinstance(charset, Charset):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	138 charset = Charset(charset)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	139 h.append(s, charset)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	140 return h
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	141
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	142
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	143
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	144 class Header:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	145 def __init__(self, s=None, charset=None,
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	146 maxlinelen=None, header_name=None,
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	147 continuation_ws=' ', errors='strict'):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	148 """Create a MIME-compliant header that can contain many character sets.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	149
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	150 Optional s is the initial header value. If None, the initial header
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	151 value is not set. You can later append to the header with .append()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	152 method calls. s may be a byte string or a Unicode string, but see the
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	153 .append() documentation for semantics.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	154
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	155 Optional charset serves two purposes: it has the same meaning as the
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	156 charset argument to the .append() method. It also sets the default
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	157 character set for all subsequent .append() calls that omit the charset
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	158 argument. If charset is not provided in the constructor, the us-ascii
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	159 charset is used both as s's initial charset and as the default for
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	160 subsequent .append() calls.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	161
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	162 The maximum line length can be specified explicit via maxlinelen. For
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	163 splitting the first line to a shorter value (to account for the field
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	164 header which isn't included in s, e.g. `Subject') pass in the name of
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	165 the field in header_name. The default maxlinelen is 76.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	166
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	167 continuation_ws must be RFC 2822 compliant folding whitespace (usually
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	168 either a space or a hard tab) which will be prepended to continuation
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	169 lines.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	170
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	171 errors is passed through to the .append() call.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	172 """
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	173 if charset is None:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	174 charset = USASCII
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	175 if not isinstance(charset, Charset):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	176 charset = Charset(charset)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	177 self._charset = charset
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	178 self._continuation_ws = continuation_ws
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	179 cws_expanded_len = len(continuation_ws.replace('\t', SPACE8))
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	180 # BAW: I believe `chunks' and `maxlinelen' should be non-public.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	181 self._chunks = []
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	182 if s is not None:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	183 self.append(s, charset, errors)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	184 if maxlinelen is None:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	185 maxlinelen = MAXLINELEN
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	186 if header_name is None:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	187 # We don't know anything about the field header so the first line
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	188 # is the same length as subsequent lines.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	189 self._firstlinelen = maxlinelen
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	190 else:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	191 # The first line should be shorter to take into account the field
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	192 # header. Also subtract off 2 extra for the colon and space.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	193 self._firstlinelen = maxlinelen - len(header_name) - 2
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	194 # Second and subsequent lines should subtract off the length in
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	195 # columns of the continuation whitespace prefix.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	196 self._maxlinelen = maxlinelen - cws_expanded_len
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	197
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	198 def __str__(self):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	199 """A synonym for self.encode()."""
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	200 return self.encode()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	201
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	202 def __unicode__(self):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	203 """Helper for the built-in unicode function."""
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	204 uchunks = []
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	205 lastcs = None
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	206 for s, charset in self._chunks:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	207 # We must preserve spaces between encoded and non-encoded word
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	208 # boundaries, which means for us we need to add a space when we go
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	209 # from a charset to None/us-ascii, or from None/us-ascii to a
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	210 # charset. Only do this for the second and subsequent chunks.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	211 nextcs = charset
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	212 if uchunks:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	213 if lastcs not in (None, 'us-ascii'):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	214 if nextcs in (None, 'us-ascii'):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	215 uchunks.append(USPACE)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	216 nextcs = None
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	217 elif nextcs not in (None, 'us-ascii'):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	218 uchunks.append(USPACE)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	219 lastcs = nextcs
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	220 uchunks.append(unicode(s, str(charset)))
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	221 return UEMPTYSTRING.join(uchunks)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	222
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	223 # Rich comparison operators for equality only. BAW: does it make sense to
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	224 # have or explicitly disable <, <=, >, >= operators?
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	225 def __eq__(self, other):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	226 # other may be a Header or a string. Both are fine so coerce
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	227 # ourselves to a string, swap the args and do another comparison.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	228 return other == self.encode()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	229
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	230 def __ne__(self, other):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	231 return not self == other
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	232
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	233 def append(self, s, charset=None, errors='strict'):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	234 """Append a string to the MIME header.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	235
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	236 Optional charset, if given, should be a Charset instance or the name
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	237 of a character set (which will be converted to a Charset instance). A
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	238 value of None (the default) means that the charset given in the
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	239 constructor is used.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	240
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	241 s may be a byte string or a Unicode string. If it is a byte string
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	242 (i.e. isinstance(s, str) is true), then charset is the encoding of
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	243 that byte string, and a UnicodeError will be raised if the string
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	244 cannot be decoded with that charset. If s is a Unicode string, then
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	245 charset is a hint specifying the character set of the characters in
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	246 the string. In this case, when producing an RFC 2822 compliant header
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	247 using RFC 2047 rules, the Unicode string will be encoded using the
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	248 following charsets in order: us-ascii, the charset hint, utf-8. The
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	249 first character set not to provoke a UnicodeError is used.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	250
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	251 Optional `errors' is passed as the third argument to any unicode() or
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	252 ustr.encode() call.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	253 """
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	254 if charset is None:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	255 charset = self._charset
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	256 elif not isinstance(charset, Charset):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	257 charset = Charset(charset)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	258 # If the charset is our faux 8bit charset, leave the string unchanged
48149 1f22de93daf8 Remove Barry's love of deprecated syntax to silence warnings in the email Brett Cannon <bcannon@gmail.com> parents: 41181 diff changeset	259 if charset != '8bit':
37078 3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	260 # We need to test that the string can be converted to unicode and
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	261 # back to a byte string, given the input and output codecs of the
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	262 # charset.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	263 if isinstance(s, str):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	264 # Possibly raise UnicodeError if the byte string can't be
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	265 # converted to a unicode with the input codec of the charset.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	266 incodec = charset.input_codec or 'us-ascii'
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	267 ustr = unicode(s, incodec, errors)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	268 # Now make sure that the unicode could be converted back to a
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	269 # byte string with the output codec, which may be different
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	270 # than the iput coded. Still, use the original byte string.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	271 outcodec = charset.output_codec or 'us-ascii'
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	272 ustr.encode(outcodec, errors)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	273 elif isinstance(s, unicode):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	274 # Now we have to be sure the unicode string can be converted
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	275 # to a byte string with a reasonable output codec. We want to
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	276 # use the byte string in the chunk.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	277 for charset in USASCII, charset, UTF8:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	278 try:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	279 outcodec = charset.output_codec or 'us-ascii'
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	280 s = s.encode(outcodec, errors)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	281 break
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	282 except UnicodeError:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	283 pass
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	284 else:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	285 assert False, 'utf-8 conversion failed'
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	286 self._chunks.append((s, charset))
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	287
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	288 def _split(self, s, charset, maxlinelen, splitchars):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	289 # Split up a header safely for use with encode_chunks.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	290 splittable = charset.to_splittable(s)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	291 encoded = charset.from_splittable(splittable, True)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	292 elen = charset.encoded_header_len(encoded)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	293 # If the line's encoded length first, just return it
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	294 if elen <= maxlinelen:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	295 return [(encoded, charset)]
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	296 # If we have undetermined raw 8bit characters sitting in a byte
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	297 # string, we really don't know what the right thing to do is. We
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	298 # can't really split it because it might be multibyte data which we
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	299 # could break if we split it between pairs. The least harm seems to
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	300 # be to not split the header at all, but that means they could go out
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	301 # longer than maxlinelen.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	302 if charset == '8bit':
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	303 return [(s, charset)]
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	304 # BAW: I'm not sure what the right test here is. What we're trying to
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	305 # do is be faithful to RFC 2822's recommendation that ($2.2.3):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	306 #
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	307 # "Note: Though structured field bodies are defined in such a way that
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	308 # folding can take place between many of the lexical tokens (and even
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	309 # within some of the lexical tokens), folding SHOULD be limited to
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	310 # placing the CRLF at higher-level syntactic breaks."
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	311 #
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	312 # For now, I can only imagine doing this when the charset is us-ascii,
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	313 # although it's possible that other charsets may also benefit from the
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	314 # higher-level syntactic breaks.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	315 elif charset == 'us-ascii':
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	316 return self._split_ascii(s, charset, maxlinelen, splitchars)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	317 # BAW: should we use encoded?
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	318 elif elen == len(s):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	319 # We can split on _maxlinelen boundaries because we know that the
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	320 # encoding won't change the size of the string
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	321 splitpnt = maxlinelen
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	322 first = charset.from_splittable(splittable[:splitpnt], False)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	323 last = charset.from_splittable(splittable[splitpnt:], False)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	324 else:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	325 # Binary search for split point
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	326 first, last = _binsplit(splittable, charset, maxlinelen)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	327 # first is of the proper length so just wrap it in the appropriate
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	328 # chrome. last must be recursively split.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	329 fsplittable = charset.to_splittable(first)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	330 fencoded = charset.from_splittable(fsplittable, True)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	331 chunk = [(fencoded, charset)]
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	332 return chunk + self._split(last, charset, self._maxlinelen, splitchars)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	333
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	334 def _split_ascii(self, s, charset, firstlen, splitchars):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	335 chunks = _split_ascii(s, firstlen, self._maxlinelen,
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	336 self._continuation_ws, splitchars)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	337 return zip(chunks, [charset]*len(chunks))
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	338
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	339 def _encode_chunks(self, newchunks, maxlinelen):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	340 # MIME-encode a header with many different charsets and/or encodings.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	341 #
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	342 # Given a list of pairs (string, charset), return a MIME-encoded
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	343 # string suitable for use in a header field. Each pair may have
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	344 # different charsets and/or encodings, and the resulting header will
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	345 # accurately reflect each setting.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	346 #
55215 a1b79778f22e Merged revisions 73952 via svnmerge from Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 48149 diff changeset	347 # Each encoding can be email.utils.QP (quoted-printable, for
a1b79778f22e Merged revisions 73952 via svnmerge from Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 48149 diff changeset	348 # ASCII-like character sets like iso-8859-1), email.utils.BASE64
37078 3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	349 # (Base64, for non-ASCII like character sets like KOI8-R and
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	350 # iso-2022-jp), or None (no encoding).
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	351 #
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	352 # Each pair will be represented on a separate line; the resulting
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	353 # string will be in the format:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	354 #
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	355 # =?charset1?q?Mar=EDa_Gonz=E1lez_Alonso?=\n
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	356 # =?charset2?b?SvxyZ2VuIEL2aW5n?="
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	357 chunks = []
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	358 for header, charset in newchunks:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	359 if not header:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	360 continue
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	361 if charset is None or charset.header_encoding is None:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	362 s = header
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	363 else:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	364 s = charset.header_encode(header)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	365 # Don't add more folding whitespace than necessary
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	366 if chunks and chunks[-1].endswith(' '):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	367 extra = ''
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	368 else:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	369 extra = ' '
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	370 _max_append(chunks, s, maxlinelen, extra)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	371 joiner = NL + self._continuation_ws
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	372 return joiner.join(chunks)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	373
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	374 def encode(self, splitchars=';, '):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	375 """Encode a message header into an RFC-compliant format.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	376
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	377 There are many issues involved in converting a given string for use in
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	378 an email header. Only certain character sets are readable in most
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	379 email clients, and as header strings can only contain a subset of
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	380 7-bit ASCII, care must be taken to properly convert and encode (with
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	381 Base64 or quoted-printable) header strings. In addition, there is a
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	382 75-character length limit on any given encoded header field, so
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	383 line-wrapping must be performed, even with double-byte character sets.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	384
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	385 This method will do its best to convert the string to the correct
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	386 character set used in email, and encode and line wrap it safely with
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	387 the appropriate scheme for that character set.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	388
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	389 If the given charset is not known or an error occurs during
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	390 conversion, this function will return the header untouched.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	391
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	392 Optional splitchars is a string containing characters to split long
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	393 ASCII lines on, in rough support of RFC 2822's `highest level
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	394 syntactic breaks'. This doesn't affect RFC 2047 encoded lines.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	395 """
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	396 newchunks = []
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	397 maxlinelen = self._firstlinelen
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	398 lastlen = 0
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	399 for s, charset in self._chunks:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	400 # The first bit of the next chunk should be just long enough to
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	401 # fill the next line. Don't forget the space separating the
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	402 # encoded words.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	403 targetlen = maxlinelen - lastlen - 1
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	404 if targetlen < charset.encoded_header_len(''):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	405 # Stick it on the next line
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	406 targetlen = maxlinelen
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	407 newchunks += self._split(s, charset, targetlen, splitchars)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	408 lastchunk, lastcharset = newchunks[-1]
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	409 lastlen = lastcharset.encoded_header_len(lastchunk)
67478 adc7e1671d8c Merged revisions 87873 via svnmerge from R. David Murray <rdmurray@bitdance.com> parents: 63709 diff changeset	410 value = self._encode_chunks(newchunks, maxlinelen)
adc7e1671d8c Merged revisions 87873 via svnmerge from R. David Murray <rdmurray@bitdance.com> parents: 63709 diff changeset	411 if _embeded_header.search(value):
adc7e1671d8c Merged revisions 87873 via svnmerge from R. David Murray <rdmurray@bitdance.com> parents: 63709 diff changeset	412 raise HeaderParseError("header value appears to contain "
adc7e1671d8c Merged revisions 87873 via svnmerge from R. David Murray <rdmurray@bitdance.com> parents: 63709 diff changeset	413 "an embedded header: {!r}".format(value))
adc7e1671d8c Merged revisions 87873 via svnmerge from R. David Murray <rdmurray@bitdance.com> parents: 63709 diff changeset	414 return value
37078 3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	415
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	416
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	417
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	418 def _split_ascii(s, firstlen, restlen, continuation_ws, splitchars):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	419 lines = []
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	420 maxlen = firstlen
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	421 for line in s.splitlines():
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	422 # Ignore any leading whitespace (i.e. continuation whitespace) already
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	423 # on the line, since we'll be adding our own.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	424 line = line.lstrip()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	425 if len(line) < maxlen:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	426 lines.append(line)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	427 maxlen = restlen
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	428 continue
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	429 # Attempt to split the line at the highest-level syntactic break
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	430 # possible. Note that we don't have a lot of smarts about field
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	431 # syntax; we just try to break on semi-colons, then commas, then
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	432 # whitespace.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	433 for ch in splitchars:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	434 if ch in line:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	435 break
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	436 else:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	437 # There's nothing useful to split the line on, not even spaces, so
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	438 # just append this line unchanged
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	439 lines.append(line)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	440 maxlen = restlen
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	441 continue
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	442 # Now split the line on the character plus trailing whitespace
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	443 cre = re.compile(r'%s\s*' % ch)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	444 if ch in ';,':
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	445 eol = ch
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	446 else:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	447 eol = ''
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	448 joiner = eol + ' '
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	449 joinlen = len(joiner)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	450 wslen = len(continuation_ws.replace('\t', SPACE8))
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	451 this = []
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	452 linelen = 0
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	453 for part in cre.split(line):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	454 curlen = linelen + max(0, len(this)-1) * joinlen
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	455 partlen = len(part)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	456 onfirstline = not lines
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	457 # We don't want to split after the field name, if we're on the
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	458 # first line and the field name is present in the header string.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	459 if ch == ' ' and onfirstline and \
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	460 len(this) == 1 and fcre.match(this[0]):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	461 this.append(part)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	462 linelen += partlen
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	463 elif curlen + partlen > maxlen:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	464 if this:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	465 lines.append(joiner.join(this) + eol)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	466 # If this part is longer than maxlen and we aren't already
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	467 # splitting on whitespace, try to recursively split this line
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	468 # on whitespace.
48149 1f22de93daf8 Remove Barry's love of deprecated syntax to silence warnings in the email Brett Cannon <bcannon@gmail.com> parents: 41181 diff changeset	469 if partlen > maxlen and ch != ' ':
37078 3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	470 subl = _split_ascii(part, maxlen, restlen,
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	471 continuation_ws, ' ')
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	472 lines.extend(subl[:-1])
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	473 this = [subl[-1]]
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	474 else:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	475 this = [part]
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	476 linelen = wslen + len(this[-1])
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	477 maxlen = restlen
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	478 else:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	479 this.append(part)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	480 linelen += partlen
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	481 # Put any left over parts on a line by themselves
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	482 if this:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	483 lines.append(joiner.join(this))
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	484 return lines
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	485
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	486
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	487
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	488 def _binsplit(splittable, charset, maxlinelen):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	489 i = 0
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	490 j = len(splittable)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	491 while i < j:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	492 # Invariants:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	493 # 1. splittable[:k] fits for all k <= i (note that we assume,
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	494 # at the start, that splittable[:0] fits).
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	495 # 2. splittable[:k] does not fit for any k > j (at the start,
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	496 # this means we shouldn't look at any k > len(splittable)).
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	497 # 3. We don't know about splittable[:k] for k in i+1..j.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	498 # 4. We want to set i to the largest k that fits, with i <= k <= j.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	499 #
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	500 m = (i+j+1) >> 1 # ceiling((i+j)/2); i < m <= j
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	501 chunk = charset.from_splittable(splittable[:m], True)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	502 chunklen = charset.encoded_header_len(chunk)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	503 if chunklen <= maxlinelen:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	504 # m is acceptable, so is a new lower bound.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	505 i = m
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	506 else:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	507 # m is not acceptable, so final i must be < m.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	508 j = m - 1
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	509 # i == j. Invariant #1 implies that splittable[:i] fits, and
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	510 # invariant #2 implies that splittable[:i+1] does not fit, so i
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	511 # is what we're looking for.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	512 first = charset.from_splittable(splittable[:i], False)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	513 last = charset.from_splittable(splittable[i:], False)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	514 return first, last

Mercurial > cpython-lru_cache-weakref

annotate Lib/email/header.py @ 80661:94de77bd0b4b 2.7