cpython-lru_cache-weakref: Lib/email/feedparser.py annotate

annotate Lib/email/feedparser.py @ 58765:2ba7677bfa95 legacy-trunk

testCopysign was defined twice in test_math; combine the definitions

author	Mark Dickinson <dickinsm@gmail.com>
date	Sat, 06 Feb 2010 23:11:25 +0000
parents	3f90a93ff5d0
children	0c5019afc55b 3d11ba7989df

rev	line source
37078 3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	1 # Copyright (C) 2004-2006 Python Software Foundation
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	2 # Authors: Baxter, Wouters and Warsaw
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	3 # Contact: email-sig@python.org
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	4
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	5 """FeedParser - An email feed parser.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	6
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	7 The feed parser implements an interface for incrementally parsing an email
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	8 message, line by line. This has advantages for certain applications, such as
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	9 those reading email messages off a socket.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	10
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	11 FeedParser.feed() is the primary interface for pushing new data into the
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	12 parser. It returns when there's nothing more it can do with the available
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	13 data. When you have no more data to push into the parser, call .close().
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	14 This completes the parsing and returns the root message object.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	15
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	16 The other advantage of this parser is that it will never throw a parsing
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	17 exception. Instead, when it finds something unexpected, it adds a 'defect' to
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	18 the current message. Defects are just instances that live on the message
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	19 object's .defects attribute.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	20 """
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	21
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	22 __all__ = ['FeedParser']
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	23
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	24 import re
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	25
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	26 from email import errors
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	27 from email import message
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	28
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	29 NLCRE = re.compile('\r\n\|\r\|\n')
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	30 NLCRE_bol = re.compile('(\r\n\|\r\|\n)')
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	31 NLCRE_eol = re.compile('(\r\n\|\r\|\n)$')
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	32 NLCRE_crack = re.compile('(\r\n\|\r\|\n)')
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	33 # RFC 2822 $3.6.8 Optional fields. ftext is %d33-57 / %d59-126, Any character
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	34 # except controls, SP, and ":".
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	35 headerRE = re.compile(r'^(From \|[\041-\071\073-\176]{1,}:\|[\t ])')
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	36 EMPTYSTRING = ''
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	37 NL = '\n'
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	38
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	39 NeedMoreData = object()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	40
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	41
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	42
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	43 class BufferedSubFile(object):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	44 """A file-ish object that can have new data loaded into it.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	45
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	46 You can also push and pop line-matching predicates onto a stack. When the
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	47 current predicate matches the current line, a false EOF response
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	48 (i.e. empty string) is returned instead. This lets the parser adhere to a
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	49 simple abstraction -- it parses until EOF closes the current message.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	50 """
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	51 def __init__(self):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	52 # The last partial line pushed into this object.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	53 self._partial = ''
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	54 # The list of full, pushed lines, in reverse order
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	55 self._lines = []
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	56 # The stack of false-EOF checking predicates.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	57 self._eofstack = []
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	58 # A flag indicating whether the file has been closed or not.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	59 self._closed = False
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	60
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	61 def push_eof_matcher(self, pred):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	62 self._eofstack.append(pred)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	63
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	64 def pop_eof_matcher(self):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	65 return self._eofstack.pop()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	66
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	67 def close(self):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	68 # Don't forget any trailing partial line.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	69 self._lines.append(self._partial)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	70 self._partial = ''
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	71 self._closed = True
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	72
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	73 def readline(self):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	74 if not self._lines:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	75 if self._closed:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	76 return ''
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	77 return NeedMoreData
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	78 # Pop the line off the stack and see if it matches the current
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	79 # false-EOF predicate.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	80 line = self._lines.pop()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	81 # RFC 2046, section 5.1.2 requires us to recognize outer level
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	82 # boundaries at any level of inner nesting. Do this, but be sure it's
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	83 # in the order of most to least nested.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	84 for ateof in self._eofstack[::-1]:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	85 if ateof(line):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	86 # We're at the false EOF. But push the last line back first.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	87 self._lines.append(line)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	88 return ''
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	89 return line
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	90
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	91 def unreadline(self, line):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	92 # Let the consumer push a line back into the buffer.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	93 assert line is not NeedMoreData
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	94 self._lines.append(line)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	95
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	96 def push(self, data):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	97 """Push some new data into this object."""
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	98 # Handle any previous leftovers
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	99 data, self._partial = self._partial + data, ''
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	100 # Crack into lines, but preserve the newlines on the end of each
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	101 parts = NLCRE_crack.split(data)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	102 # The ahem interesting behaviour of re.split when supplied grouping
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	103 # parentheses is that the last element of the resulting list is the
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	104 # data after the final RE. In the case of a NL/CR terminated string,
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	105 # this is the empty string.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	106 self._partial = parts.pop()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	107 # parts is a list of strings, alternating between the line contents
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	108 # and the eol character(s). Gather up a list of lines after
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	109 # re-attaching the newlines.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	110 lines = []
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	111 for i in range(len(parts) // 2):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	112 lines.append(parts[i2] + parts[i2+1])
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	113 self.pushlines(lines)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	114
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	115 def pushlines(self, lines):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	116 # Reverse and insert at the front of the lines.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	117 self._lines[:0] = lines[::-1]
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	118
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	119 def is_closed(self):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	120 return self._closed
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	121
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	122 def __iter__(self):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	123 return self
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	124
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	125 def next(self):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	126 line = self.readline()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	127 if line == '':
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	128 raise StopIteration
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	129 return line
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	130
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	131
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	132
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	133 class FeedParser:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	134 """A feed-style parser of email."""
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	135
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	136 def __init__(self, _factory=message.Message):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	137 """_factory is called with no arguments to create a new message obj"""
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	138 self._factory = _factory
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	139 self._input = BufferedSubFile()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	140 self._msgstack = []
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	141 self._parse = self._parsegen().next
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	142 self._cur = None
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	143 self._last = None
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	144 self._headersonly = False
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	145
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	146 # Non-public interface for supporting Parser's headersonly flag
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	147 def _set_headersonly(self):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	148 self._headersonly = True
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	149
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	150 def feed(self, data):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	151 """Push more data into the parser."""
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	152 self._input.push(data)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	153 self._call_parse()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	154
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	155 def _call_parse(self):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	156 try:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	157 self._parse()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	158 except StopIteration:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	159 pass
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	160
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	161 def close(self):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	162 """Parse all remaining data and return the root message object."""
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	163 self._input.close()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	164 self._call_parse()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	165 root = self._pop_message()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	166 assert not self._msgstack
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	167 # Look for final set of defects
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	168 if root.get_content_maintype() == 'multipart' \
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	169 and not root.is_multipart():
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	170 root.defects.append(errors.MultipartInvariantViolationDefect())
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	171 return root
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	172
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	173 def _new_message(self):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	174 msg = self._factory()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	175 if self._cur and self._cur.get_content_type() == 'multipart/digest':
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	176 msg.set_default_type('message/rfc822')
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	177 if self._msgstack:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	178 self._msgstack[-1].attach(msg)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	179 self._msgstack.append(msg)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	180 self._cur = msg
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	181 self._last = msg
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	182
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	183 def _pop_message(self):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	184 retval = self._msgstack.pop()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	185 if self._msgstack:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	186 self._cur = self._msgstack[-1]
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	187 else:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	188 self._cur = None
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	189 return retval
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	190
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	191 def _parsegen(self):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	192 # Create a new message and start by parsing headers.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	193 self._new_message()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	194 headers = []
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	195 # Collect the headers, searching for a line that doesn't match the RFC
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	196 # 2822 header or continuation pattern (including an empty line).
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	197 for line in self._input:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	198 if line is NeedMoreData:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	199 yield NeedMoreData
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	200 continue
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	201 if not headerRE.match(line):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	202 # If we saw the RFC defined header/body separator
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	203 # (i.e. newline), just throw it away. Otherwise the line is
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	204 # part of the body so push it back.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	205 if not NLCRE.match(line):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	206 self._input.unreadline(line)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	207 break
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	208 headers.append(line)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	209 # Done with the headers, so parse them and figure out what we're
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	210 # supposed to see in the body of the message.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	211 self._parse_headers(headers)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	212 # Headers-only parsing is a backwards compatibility hack, which was
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	213 # necessary in the older parser, which could throw errors. All
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	214 # remaining lines in the input are thrown into the message body.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	215 if self._headersonly:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	216 lines = []
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	217 while True:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	218 line = self._input.readline()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	219 if line is NeedMoreData:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	220 yield NeedMoreData
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	221 continue
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	222 if line == '':
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	223 break
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	224 lines.append(line)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	225 self._cur.set_payload(EMPTYSTRING.join(lines))
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	226 return
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	227 if self._cur.get_content_type() == 'message/delivery-status':
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	228 # message/delivery-status contains blocks of headers separated by
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	229 # a blank line. We'll represent each header block as a separate
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	230 # nested message object, but the processing is a bit different
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	231 # than standard message/* types because there is no body for the
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	232 # nested messages. A blank line separates the subparts.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	233 while True:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	234 self._input.push_eof_matcher(NLCRE.match)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	235 for retval in self._parsegen():
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	236 if retval is NeedMoreData:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	237 yield NeedMoreData
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	238 continue
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	239 break
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	240 msg = self._pop_message()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	241 # We need to pop the EOF matcher in order to tell if we're at
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	242 # the end of the current file, not the end of the last block
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	243 # of message headers.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	244 self._input.pop_eof_matcher()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	245 # The input stream must be sitting at the newline or at the
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	246 # EOF. We want to see if we're at the end of this subpart, so
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	247 # first consume the blank line, then test the next line to see
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	248 # if we're at this subpart's EOF.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	249 while True:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	250 line = self._input.readline()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	251 if line is NeedMoreData:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	252 yield NeedMoreData
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	253 continue
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	254 break
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	255 while True:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	256 line = self._input.readline()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	257 if line is NeedMoreData:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	258 yield NeedMoreData
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	259 continue
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	260 break
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	261 if line == '':
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	262 break
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	263 # Not at EOF so this is a line we're going to need.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	264 self._input.unreadline(line)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	265 return
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	266 if self._cur.get_content_maintype() == 'message':
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	267 # The message claims to be a message/* type, then what follows is
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	268 # another RFC 2822 message.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	269 for retval in self._parsegen():
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	270 if retval is NeedMoreData:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	271 yield NeedMoreData
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	272 continue
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	273 break
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	274 self._pop_message()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	275 return
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	276 if self._cur.get_content_maintype() == 'multipart':
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	277 boundary = self._cur.get_boundary()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	278 if boundary is None:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	279 # The message /claims/ to be a multipart but it has not
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	280 # defined a boundary. That's a problem which we'll handle by
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	281 # reading everything until the EOF and marking the message as
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	282 # defective.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	283 self._cur.defects.append(errors.NoBoundaryInMultipartDefect())
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	284 lines = []
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	285 for line in self._input:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	286 if line is NeedMoreData:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	287 yield NeedMoreData
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	288 continue
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	289 lines.append(line)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	290 self._cur.set_payload(EMPTYSTRING.join(lines))
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	291 return
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	292 # Create a line match predicate which matches the inter-part
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	293 # boundary as well as the end-of-multipart boundary. Don't push
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	294 # this onto the input stream until we've scanned past the
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	295 # preamble.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	296 separator = '--' + boundary
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	297 boundaryre = re.compile(
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	298 '(?P<sep>' + re.escape(separator) +
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	299 r')(?P<end>--)?(?P<ws>[ \t]*)(?P<linesep>\r\n\|\r\|\n)?$')
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	300 capturing_preamble = True
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	301 preamble = []
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	302 linesep = False
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	303 while True:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	304 line = self._input.readline()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	305 if line is NeedMoreData:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	306 yield NeedMoreData
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	307 continue
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	308 if line == '':
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	309 break
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	310 mo = boundaryre.match(line)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	311 if mo:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	312 # If we're looking at the end boundary, we're done with
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	313 # this multipart. If there was a newline at the end of
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	314 # the closing boundary, then we need to initialize the
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	315 # epilogue with the empty string (see below).
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	316 if mo.group('end'):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	317 linesep = mo.group('linesep')
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	318 break
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	319 # We saw an inter-part boundary. Were we in the preamble?
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	320 if capturing_preamble:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	321 if preamble:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	322 # According to RFC 2046, the last newline belongs
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	323 # to the boundary.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	324 lastline = preamble[-1]
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	325 eolmo = NLCRE_eol.search(lastline)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	326 if eolmo:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	327 preamble[-1] = lastline[:-len(eolmo.group(0))]
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	328 self._cur.preamble = EMPTYSTRING.join(preamble)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	329 capturing_preamble = False
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	330 self._input.unreadline(line)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	331 continue
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	332 # We saw a boundary separating two parts. Consume any
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	333 # multiple boundary lines that may be following. Our
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	334 # interpretation of RFC 2046 BNF grammar does not produce
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	335 # body parts within such double boundaries.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	336 while True:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	337 line = self._input.readline()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	338 if line is NeedMoreData:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	339 yield NeedMoreData
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	340 continue
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	341 mo = boundaryre.match(line)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	342 if not mo:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	343 self._input.unreadline(line)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	344 break
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	345 # Recurse to parse this subpart; the input stream points
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	346 # at the subpart's first line.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	347 self._input.push_eof_matcher(boundaryre.match)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	348 for retval in self._parsegen():
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	349 if retval is NeedMoreData:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	350 yield NeedMoreData
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	351 continue
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	352 break
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	353 # Because of RFC 2046, the newline preceding the boundary
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	354 # separator actually belongs to the boundary, not the
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	355 # previous subpart's payload (or epilogue if the previous
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	356 # part is a multipart).
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	357 if self._last.get_content_maintype() == 'multipart':
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	358 epilogue = self._last.epilogue
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	359 if epilogue == '':
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	360 self._last.epilogue = None
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	361 elif epilogue is not None:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	362 mo = NLCRE_eol.search(epilogue)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	363 if mo:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	364 end = len(mo.group(0))
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	365 self._last.epilogue = epilogue[:-end]
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	366 else:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	367 payload = self._last.get_payload()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	368 if isinstance(payload, basestring):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	369 mo = NLCRE_eol.search(payload)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	370 if mo:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	371 payload = payload[:-len(mo.group(0))]
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	372 self._last.set_payload(payload)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	373 self._input.pop_eof_matcher()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	374 self._pop_message()
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	375 # Set the multipart up for newline cleansing, which will
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	376 # happen if we're in a nested multipart.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	377 self._last = self._cur
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	378 else:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	379 # I think we must be in the preamble
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	380 assert capturing_preamble
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	381 preamble.append(line)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	382 # We've seen either the EOF or the end boundary. If we're still
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	383 # capturing the preamble, we never saw the start boundary. Note
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	384 # that as a defect and store the captured text as the payload.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	385 # Everything from here to the EOF is epilogue.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	386 if capturing_preamble:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	387 self._cur.defects.append(errors.StartBoundaryNotFoundDefect())
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	388 self._cur.set_payload(EMPTYSTRING.join(preamble))
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	389 epilogue = []
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	390 for line in self._input:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	391 if line is NeedMoreData:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	392 yield NeedMoreData
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	393 continue
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	394 self._cur.epilogue = EMPTYSTRING.join(epilogue)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	395 return
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	396 # If the end boundary ended in a newline, we'll need to make sure
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	397 # the epilogue isn't None
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	398 if linesep:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	399 epilogue = ['']
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	400 else:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	401 epilogue = []
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	402 for line in self._input:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	403 if line is NeedMoreData:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	404 yield NeedMoreData
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	405 continue
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	406 epilogue.append(line)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	407 # Any CRLF at the front of the epilogue is not technically part of
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	408 # the epilogue. Also, watch out for an empty string epilogue,
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	409 # which means a single newline.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	410 if epilogue:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	411 firstline = epilogue[0]
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	412 bolmo = NLCRE_bol.match(firstline)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	413 if bolmo:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	414 epilogue[0] = firstline[len(bolmo.group(0)):]
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	415 self._cur.epilogue = EMPTYSTRING.join(epilogue)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	416 return
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	417 # Otherwise, it's some non-multipart type, so the entire rest of the
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	418 # file contents becomes the payload.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	419 lines = []
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	420 for line in self._input:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	421 if line is NeedMoreData:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	422 yield NeedMoreData
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	423 continue
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	424 lines.append(line)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	425 self._cur.set_payload(EMPTYSTRING.join(lines))
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	426
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	427 def _parse_headers(self, lines):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	428 # Passed a list of lines that make up the headers for the current msg
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	429 lastheader = ''
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	430 lastvalue = []
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	431 for lineno, line in enumerate(lines):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	432 # Check for continuation
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	433 if line[0] in ' \t':
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	434 if not lastheader:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	435 # The first line of the headers was a continuation. This
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	436 # is illegal, so let's note the defect, store the illegal
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	437 # line, and ignore it for purposes of headers.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	438 defect = errors.FirstHeaderLineIsContinuationDefect(line)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	439 self._cur.defects.append(defect)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	440 continue
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	441 lastvalue.append(line)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	442 continue
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	443 if lastheader:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	444 # XXX reconsider the joining of folded lines
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	445 lhdr = EMPTYSTRING.join(lastvalue)[:-1].rstrip('\r\n')
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	446 self._cur[lastheader] = lhdr
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	447 lastheader, lastvalue = '', []
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	448 # Check for envelope header, i.e. unix-from
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	449 if line.startswith('From '):
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	450 if lineno == 0:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	451 # Strip off the trailing newline
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	452 mo = NLCRE_eol.search(line)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	453 if mo:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	454 line = line[:-len(mo.group(0))]
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	455 self._cur.set_unixfrom(line)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	456 continue
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	457 elif lineno == len(lines) - 1:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	458 # Something looking like a unix-from at the end - it's
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	459 # probably the first line of the body, so push back the
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	460 # line and stop.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	461 self._input.unreadline(line)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	462 return
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	463 else:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	464 # Weirdly placed unix-from line. Note this as a defect
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	465 # and ignore it.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	466 defect = errors.MisplacedEnvelopeHeaderDefect(line)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	467 self._cur.defects.append(defect)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	468 continue
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	469 # Split the line on the colon separating field name from value.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	470 i = line.find(':')
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	471 if i < 0:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	472 defect = errors.MalformedHeaderDefect(line)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	473 self._cur.defects.append(defect)
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	474 continue
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	475 lastheader = line[:i]
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	476 lastvalue = [line[i+1:].lstrip()]
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	477 # Done with all the lines, so handle the last header.
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	478 if lastheader:
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	479 # XXX reconsider the joining of folded lines
3f90a93ff5d0 Merge email package 4.0 from the sandbox, including documentation, test cases, Barry Warsaw <barry@python.org> parents: diff changeset	480 self._cur[lastheader] = EMPTYSTRING.join(lastvalue).rstrip('\r\n')

Mercurial > cpython-lru_cache-weakref

annotate Lib/email/feedparser.py @ 58765:2ba7677bfa95 legacy-trunk