cpython-lru_cache-weakref: Lib/email/feedparser.py annotate

annotate Lib/email/feedparser.py @ 80661:94de77bd0b4b 2.7

Issue #16194: document imp.load_dynamic problems when read several extension modules from the same binary object file.

author	Andrew Svetlov <andrew.svetlov@gmail.com>
date	Thu, 29 Nov 2012 19:51:28 +0200
parents	01711f60fffc
children	8c2635afbfe1

rev	line source
37078 3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	1 # Copyright (C) 2004-2006 Python Software Foundation
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	2 # Authors: Baxter, Wouters and Warsaw
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	3 # Contact: email-sig@python.org
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	4
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	5 """FeedParser - An email feed parser.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	6
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	7 The feed parser implements an interface for incrementally parsing an email
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	8 message, line by line. This has advantages for certain applications, such as
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	9 those reading email messages off a socket.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	10
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	11 FeedParser.feed() is the primary interface for pushing new data into the
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	12 parser. It returns when there's nothing more it can do with the available
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	13 data. When you have no more data to push into the parser, call .close().
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	14 This completes the parsing and returns the root message object.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	15
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	16 The other advantage of this parser is that it will never throw a parsing
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	17 exception. Instead, when it finds something unexpected, it adds a 'defect' to
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	18 the current message. Defects are just instances that live on the message
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	19 object's .defects attribute.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	20 """
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	21
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	22 __all__ = ['FeedParser']
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	23
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	24 import re
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	25
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	26 from email import errors
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	27 from email import message
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	28
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	29 NLCRE = re.compile('\r\n\|\r\|\n')
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	30 NLCRE_bol = re.compile('(\r\n\|\r\|\n)')
62052 3d11ba7989df #5610: use \Z not $ so we don't eat extra chars when body part ends with \r\n. R. David Murray <rdmurray@bitdance.com> parents: 37078 diff changeset	31 NLCRE_eol = re.compile('(\r\n\|\r\|\n)\Z')
37078 3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	32 NLCRE_crack = re.compile('(\r\n\|\r\|\n)')
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	33 # RFC 2822 $3.6.8 Optional fields. ftext is %d33-57 / %d59-126, Any character
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	34 # except controls, SP, and ":".
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	35 headerRE = re.compile(r'^(From \|[\041-\071\073-\176]{1,}:\|[\t ])')
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	36 EMPTYSTRING = ''
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	37 NL = '\n'
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	38
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	39 NeedMoreData = object()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	40
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	41
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	42
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	43 class BufferedSubFile(object):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	44 """A file-ish object that can have new data loaded into it.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	45
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	46 You can also push and pop line-matching predicates onto a stack. When the
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	47 current predicate matches the current line, a false EOF response
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	48 (i.e. empty string) is returned instead. This lets the parser adhere to a
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	49 simple abstraction -- it parses until EOF closes the current message.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	50 """
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	51 def __init__(self):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	52 # The last partial line pushed into this object.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	53 self._partial = ''
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	54 # The list of full, pushed lines, in reverse order
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	55 self._lines = []
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	56 # The stack of false-EOF checking predicates.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	57 self._eofstack = []
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	58 # A flag indicating whether the file has been closed or not.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	59 self._closed = False
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	60
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	61 def push_eof_matcher(self, pred):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	62 self._eofstack.append(pred)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	63
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	64 def pop_eof_matcher(self):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	65 return self._eofstack.pop()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	66
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	67 def close(self):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	68 # Don't forget any trailing partial line.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	69 self._lines.append(self._partial)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	70 self._partial = ''
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	71 self._closed = True
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	72
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	73 def readline(self):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	74 if not self._lines:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	75 if self._closed:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	76 return ''
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	77 return NeedMoreData
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	78 # Pop the line off the stack and see if it matches the current
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	79 # false-EOF predicate.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	80 line = self._lines.pop()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	81 # RFC 2046, section 5.1.2 requires us to recognize outer level
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	82 # boundaries at any level of inner nesting. Do this, but be sure it's
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	83 # in the order of most to least nested.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	84 for ateof in self._eofstack[::-1]:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	85 if ateof(line):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	86 # We're at the false EOF. But push the last line back first.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	87 self._lines.append(line)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	88 return ''
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	89 return line
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	90
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	91 def unreadline(self, line):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	92 # Let the consumer push a line back into the buffer.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	93 assert line is not NeedMoreData
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	94 self._lines.append(line)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	95
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	96 def push(self, data):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	97 """Push some new data into this object."""
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	98 # Handle any previous leftovers
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	99 data, self._partial = self._partial + data, ''
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	100 # Crack into lines, but preserve the newlines on the end of each
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	101 parts = NLCRE_crack.split(data)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	102 # The ahem interesting behaviour of re.split when supplied grouping
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	103 # parentheses is that the last element of the resulting list is the
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	104 # data after the final RE. In the case of a NL/CR terminated string,
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	105 # this is the empty string.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	106 self._partial = parts.pop()
63049 01711f60fffc Merged revisions 82922 via svnmerge from R. David Murray <rdmurray@bitdance.com> parents: 62052 diff changeset	107 #GAN 29Mar09 bugs 1555570, 1721862 Confusion at 8K boundary ending with \r:
01711f60fffc Merged revisions 82922 via svnmerge from R. David Murray <rdmurray@bitdance.com> parents: 62052 diff changeset	108 # is there a \n to follow later?
01711f60fffc Merged revisions 82922 via svnmerge from R. David Murray <rdmurray@bitdance.com> parents: 62052 diff changeset	109 if not self._partial and parts and parts[-1].endswith('\r'):
01711f60fffc Merged revisions 82922 via svnmerge from R. David Murray <rdmurray@bitdance.com> parents: 62052 diff changeset	110 self._partial = parts.pop(-2)+parts.pop()
37078 3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	111 # parts is a list of strings, alternating between the line contents
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	112 # and the eol character(s). Gather up a list of lines after
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	113 # re-attaching the newlines.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	114 lines = []
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	115 for i in range(len(parts) // 2):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	116 lines.append(parts[i2] + parts[i2+1])
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	117 self.pushlines(lines)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	118
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	119 def pushlines(self, lines):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	120 # Reverse and insert at the front of the lines.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	121 self._lines[:0] = lines[::-1]
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	122
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	123 def is_closed(self):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	124 return self._closed
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	125
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	126 def __iter__(self):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	127 return self
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	128
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	129 def next(self):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	130 line = self.readline()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	131 if line == '':
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	132 raise StopIteration
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	133 return line
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	134
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	135
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	136
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	137 class FeedParser:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	138 """A feed-style parser of email."""
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	139
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	140 def __init__(self, _factory=message.Message):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	141 """_factory is called with no arguments to create a new message obj"""
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	142 self._factory = _factory
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	143 self._input = BufferedSubFile()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	144 self._msgstack = []
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	145 self._parse = self._parsegen().next
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	146 self._cur = None
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	147 self._last = None
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	148 self._headersonly = False
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	149
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	150 # Non-public interface for supporting Parser's headersonly flag
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	151 def _set_headersonly(self):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	152 self._headersonly = True
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	153
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	154 def feed(self, data):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	155 """Push more data into the parser."""
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	156 self._input.push(data)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	157 self._call_parse()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	158
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	159 def _call_parse(self):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	160 try:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	161 self._parse()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	162 except StopIteration:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	163 pass
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	164
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	165 def close(self):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	166 """Parse all remaining data and return the root message object."""
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	167 self._input.close()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	168 self._call_parse()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	169 root = self._pop_message()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	170 assert not self._msgstack
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	171 # Look for final set of defects
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	172 if root.get_content_maintype() == 'multipart' \
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	173 and not root.is_multipart():
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	174 root.defects.append(errors.MultipartInvariantViolationDefect())
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	175 return root
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	176
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	177 def _new_message(self):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	178 msg = self._factory()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	179 if self._cur and self._cur.get_content_type() == 'multipart/digest':
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	180 msg.set_default_type('message/rfc822')
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	181 if self._msgstack:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	182 self._msgstack[-1].attach(msg)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	183 self._msgstack.append(msg)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	184 self._cur = msg
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	185 self._last = msg
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	186
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	187 def _pop_message(self):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	188 retval = self._msgstack.pop()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	189 if self._msgstack:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	190 self._cur = self._msgstack[-1]
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	191 else:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	192 self._cur = None
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	193 return retval
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	194
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	195 def _parsegen(self):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	196 # Create a new message and start by parsing headers.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	197 self._new_message()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	198 headers = []
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	199 # Collect the headers, searching for a line that doesn't match the RFC
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	200 # 2822 header or continuation pattern (including an empty line).
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	201 for line in self._input:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	202 if line is NeedMoreData:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	203 yield NeedMoreData
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	204 continue
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	205 if not headerRE.match(line):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	206 # If we saw the RFC defined header/body separator
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	207 # (i.e. newline), just throw it away. Otherwise the line is
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	208 # part of the body so push it back.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	209 if not NLCRE.match(line):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	210 self._input.unreadline(line)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	211 break
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	212 headers.append(line)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	213 # Done with the headers, so parse them and figure out what we're
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	214 # supposed to see in the body of the message.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	215 self._parse_headers(headers)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	216 # Headers-only parsing is a backwards compatibility hack, which was
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	217 # necessary in the older parser, which could throw errors. All
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	218 # remaining lines in the input are thrown into the message body.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	219 if self._headersonly:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	220 lines = []
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	221 while True:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	222 line = self._input.readline()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	223 if line is NeedMoreData:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	224 yield NeedMoreData
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	225 continue
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	226 if line == '':
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	227 break
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	228 lines.append(line)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	229 self._cur.set_payload(EMPTYSTRING.join(lines))
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	230 return
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	231 if self._cur.get_content_type() == 'message/delivery-status':
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	232 # message/delivery-status contains blocks of headers separated by
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	233 # a blank line. We'll represent each header block as a separate
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	234 # nested message object, but the processing is a bit different
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	235 # than standard message/* types because there is no body for the
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	236 # nested messages. A blank line separates the subparts.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	237 while True:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	238 self._input.push_eof_matcher(NLCRE.match)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	239 for retval in self._parsegen():
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	240 if retval is NeedMoreData:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	241 yield NeedMoreData
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	242 continue
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	243 break
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	244 msg = self._pop_message()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	245 # We need to pop the EOF matcher in order to tell if we're at
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	246 # the end of the current file, not the end of the last block
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	247 # of message headers.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	248 self._input.pop_eof_matcher()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	249 # The input stream must be sitting at the newline or at the
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	250 # EOF. We want to see if we're at the end of this subpart, so
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	251 # first consume the blank line, then test the next line to see
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	252 # if we're at this subpart's EOF.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	253 while True:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	254 line = self._input.readline()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	255 if line is NeedMoreData:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	256 yield NeedMoreData
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	257 continue
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	258 break
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	259 while True:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	260 line = self._input.readline()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	261 if line is NeedMoreData:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	262 yield NeedMoreData
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	263 continue
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	264 break
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	265 if line == '':
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	266 break
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	267 # Not at EOF so this is a line we're going to need.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	268 self._input.unreadline(line)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	269 return
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	270 if self._cur.get_content_maintype() == 'message':
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	271 # The message claims to be a message/* type, then what follows is
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	272 # another RFC 2822 message.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	273 for retval in self._parsegen():
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	274 if retval is NeedMoreData:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	275 yield NeedMoreData
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	276 continue
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	277 break
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	278 self._pop_message()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	279 return
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	280 if self._cur.get_content_maintype() == 'multipart':
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	281 boundary = self._cur.get_boundary()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	282 if boundary is None:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	283 # The message /claims/ to be a multipart but it has not
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	284 # defined a boundary. That's a problem which we'll handle by
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	285 # reading everything until the EOF and marking the message as
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	286 # defective.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	287 self._cur.defects.append(errors.NoBoundaryInMultipartDefect())
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	288 lines = []
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	289 for line in self._input:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	290 if line is NeedMoreData:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	291 yield NeedMoreData
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	292 continue
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	293 lines.append(line)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	294 self._cur.set_payload(EMPTYSTRING.join(lines))
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	295 return
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	296 # Create a line match predicate which matches the inter-part
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	297 # boundary as well as the end-of-multipart boundary. Don't push
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	298 # this onto the input stream until we've scanned past the
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	299 # preamble.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	300 separator = '--' + boundary
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	301 boundaryre = re.compile(
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	302 '(?P<sep>' + re.escape(separator) +
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	303 r')(?P<end>--)?(?P<ws>[ \t]*)(?P<linesep>\r\n\|\r\|\n)?$')
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	304 capturing_preamble = True
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	305 preamble = []
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	306 linesep = False
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	307 while True:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	308 line = self._input.readline()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	309 if line is NeedMoreData:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	310 yield NeedMoreData
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	311 continue
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	312 if line == '':
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	313 break
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	314 mo = boundaryre.match(line)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	315 if mo:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	316 # If we're looking at the end boundary, we're done with
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	317 # this multipart. If there was a newline at the end of
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	318 # the closing boundary, then we need to initialize the
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	319 # epilogue with the empty string (see below).
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	320 if mo.group('end'):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	321 linesep = mo.group('linesep')
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	322 break
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	323 # We saw an inter-part boundary. Were we in the preamble?
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	324 if capturing_preamble:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	325 if preamble:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	326 # According to RFC 2046, the last newline belongs
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	327 # to the boundary.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	328 lastline = preamble[-1]
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	329 eolmo = NLCRE_eol.search(lastline)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	330 if eolmo:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	331 preamble[-1] = lastline[:-len(eolmo.group(0))]
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	332 self._cur.preamble = EMPTYSTRING.join(preamble)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	333 capturing_preamble = False
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	334 self._input.unreadline(line)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	335 continue
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	336 # We saw a boundary separating two parts. Consume any
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	337 # multiple boundary lines that may be following. Our
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	338 # interpretation of RFC 2046 BNF grammar does not produce
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	339 # body parts within such double boundaries.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	340 while True:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	341 line = self._input.readline()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	342 if line is NeedMoreData:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	343 yield NeedMoreData
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	344 continue
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	345 mo = boundaryre.match(line)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	346 if not mo:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	347 self._input.unreadline(line)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	348 break
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	349 # Recurse to parse this subpart; the input stream points
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	350 # at the subpart's first line.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	351 self._input.push_eof_matcher(boundaryre.match)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	352 for retval in self._parsegen():
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	353 if retval is NeedMoreData:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	354 yield NeedMoreData
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	355 continue
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	356 break
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	357 # Because of RFC 2046, the newline preceding the boundary
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	358 # separator actually belongs to the boundary, not the
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	359 # previous subpart's payload (or epilogue if the previous
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	360 # part is a multipart).
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	361 if self._last.get_content_maintype() == 'multipart':
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	362 epilogue = self._last.epilogue
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	363 if epilogue == '':
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	364 self._last.epilogue = None
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	365 elif epilogue is not None:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	366 mo = NLCRE_eol.search(epilogue)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	367 if mo:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	368 end = len(mo.group(0))
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	369 self._last.epilogue = epilogue[:-end]
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	370 else:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	371 payload = self._last.get_payload()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	372 if isinstance(payload, basestring):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	373 mo = NLCRE_eol.search(payload)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	374 if mo:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	375 payload = payload[:-len(mo.group(0))]
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	376 self._last.set_payload(payload)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	377 self._input.pop_eof_matcher()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	378 self._pop_message()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	379 # Set the multipart up for newline cleansing, which will
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	380 # happen if we're in a nested multipart.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	381 self._last = self._cur
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	382 else:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	383 # I think we must be in the preamble
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	384 assert capturing_preamble
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	385 preamble.append(line)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	386 # We've seen either the EOF or the end boundary. If we're still
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	387 # capturing the preamble, we never saw the start boundary. Note
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	388 # that as a defect and store the captured text as the payload.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	389 # Everything from here to the EOF is epilogue.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	390 if capturing_preamble:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	391 self._cur.defects.append(errors.StartBoundaryNotFoundDefect())
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	392 self._cur.set_payload(EMPTYSTRING.join(preamble))
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	393 epilogue = []
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	394 for line in self._input:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	395 if line is NeedMoreData:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	396 yield NeedMoreData
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	397 continue
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	398 self._cur.epilogue = EMPTYSTRING.join(epilogue)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	399 return
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	400 # If the end boundary ended in a newline, we'll need to make sure
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	401 # the epilogue isn't None
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	402 if linesep:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	403 epilogue = ['']
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	404 else:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	405 epilogue = []
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	406 for line in self._input:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	407 if line is NeedMoreData:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	408 yield NeedMoreData
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	409 continue
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	410 epilogue.append(line)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	411 # Any CRLF at the front of the epilogue is not technically part of
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	412 # the epilogue. Also, watch out for an empty string epilogue,
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	413 # which means a single newline.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	414 if epilogue:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	415 firstline = epilogue[0]
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	416 bolmo = NLCRE_bol.match(firstline)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	417 if bolmo:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	418 epilogue[0] = firstline[len(bolmo.group(0)):]
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	419 self._cur.epilogue = EMPTYSTRING.join(epilogue)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	420 return
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	421 # Otherwise, it's some non-multipart type, so the entire rest of the
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	422 # file contents becomes the payload.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	423 lines = []
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	424 for line in self._input:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	425 if line is NeedMoreData:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	426 yield NeedMoreData
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	427 continue
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	428 lines.append(line)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	429 self._cur.set_payload(EMPTYSTRING.join(lines))
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	430
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	431 def _parse_headers(self, lines):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	432 # Passed a list of lines that make up the headers for the current msg
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	433 lastheader = ''
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	434 lastvalue = []
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	435 for lineno, line in enumerate(lines):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	436 # Check for continuation
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	437 if line[0] in ' \t':
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	438 if not lastheader:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	439 # The first line of the headers was a continuation. This
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	440 # is illegal, so let's note the defect, store the illegal
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	441 # line, and ignore it for purposes of headers.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	442 defect = errors.FirstHeaderLineIsContinuationDefect(line)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	443 self._cur.defects.append(defect)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	444 continue
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	445 lastvalue.append(line)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	446 continue
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	447 if lastheader:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	448 # XXX reconsider the joining of folded lines
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	449 lhdr = EMPTYSTRING.join(lastvalue)[:-1].rstrip('\r\n')
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	450 self._cur[lastheader] = lhdr
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	451 lastheader, lastvalue = '', []
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	452 # Check for envelope header, i.e. unix-from
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	453 if line.startswith('From '):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	454 if lineno == 0:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	455 # Strip off the trailing newline
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	456 mo = NLCRE_eol.search(line)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	457 if mo:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	458 line = line[:-len(mo.group(0))]
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	459 self._cur.set_unixfrom(line)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	460 continue
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	461 elif lineno == len(lines) - 1:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	462 # Something looking like a unix-from at the end - it's
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	463 # probably the first line of the body, so push back the
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	464 # line and stop.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	465 self._input.unreadline(line)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	466 return
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	467 else:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	468 # Weirdly placed unix-from line. Note this as a defect
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	469 # and ignore it.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	470 defect = errors.MisplacedEnvelopeHeaderDefect(line)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	471 self._cur.defects.append(defect)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	472 continue
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	473 # Split the line on the colon separating field name from value.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	474 i = line.find(':')
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	475 if i < 0:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	476 defect = errors.MalformedHeaderDefect(line)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	477 self._cur.defects.append(defect)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	478 continue
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	479 lastheader = line[:i]
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	480 lastvalue = [line[i+1:].lstrip()]
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	481 # Done with all the lines, so handle the last header.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	482 if lastheader:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	483 # XXX reconsider the joining of folded lines
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	484 self._cur[lastheader] = EMPTYSTRING.join(lastvalue).rstrip('\r\n')

Mercurial > cpython-lru_cache-weakref

annotate Lib/email/feedparser.py @ 80661:94de77bd0b4b 2.7