cpython-lru_cache-weakref: Lib/email/feedparser.py annotate

annotate Lib/email/feedparser.py @ 79691:05c8928362ee

Merge __next__ method link fixes with 3.3.

author	Ezio Melotti <ezio.melotti@gmail.com>
date	Fri, 12 Oct 2012 13:46:37 +0300
parents	81e008f13b4f
children	15a391919deb

rev	line source
43019 d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	1 # Copyright (C) 2004-2006 Python Software Foundation
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	2 # Authors: Baxter, Wouters and Warsaw
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	3 # Contact: email-sig@python.org
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	4
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	5 """FeedParser - An email feed parser.
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	6
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	7 The feed parser implements an interface for incrementally parsing an email
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	8 message, line by line. This has advantages for certain applications, such as
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	9 those reading email messages off a socket.
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	10
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	11 FeedParser.feed() is the primary interface for pushing new data into the
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	12 parser. It returns when there's nothing more it can do with the available
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	13 data. When you have no more data to push into the parser, call .close().
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	14 This completes the parsing and returns the root message object.
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	15
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	16 The other advantage of this parser is that it will never throw a parsing
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	17 exception. Instead, when it finds something unexpected, it adds a 'defect' to
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	18 the current message. Defects are just instances that live on the message
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	19 object's .defects attribute.
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	20 """
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	21
75769 63a6d35fcac8 #11686: add missing entries to email __all__ lists. R David Murray <rdmurray@bitdance.com> parents: 69171 diff changeset	22 __all__ = ['FeedParser', 'BytesFeedParser']
43019 d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	23
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	24 import re
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	25
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	26 from email import errors
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	27 from email import message
77146 9388c671d52d #14731: refactor email policy framework. R David Murray <rdmurray@bitdance.com> parents: 75770 diff changeset	28 from email._policybase import compat32
43019 d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	29
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	30 NLCRE = re.compile('\r\n\|\r\|\n')
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	31 NLCRE_bol = re.compile('(\r\n\|\r\|\n)')
62347 58413676befb Merged revisions 81675 via svnmerge from R. David Murray <rdmurray@bitdance.com> parents: 43577 diff changeset	32 NLCRE_eol = re.compile('(\r\n\|\r\|\n)\Z')
43019 d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	33 NLCRE_crack = re.compile('(\r\n\|\r\|\n)')
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	34 # RFC 2822 $3.6.8 Optional fields. ftext is %d33-57 / %d59-126, Any character
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	35 # except controls, SP, and ":".
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	36 headerRE = re.compile(r'^(From \|[\041-\071\073-\176]{1,}:\|[\t ])')
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	37 EMPTYSTRING = ''
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	38 NL = '\n'
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	39
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	40 NeedMoreData = object()
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	41
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	42
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	43
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	44 class BufferedSubFile(object):
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	45 """A file-ish object that can have new data loaded into it.
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	46
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	47 You can also push and pop line-matching predicates onto a stack. When the
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	48 current predicate matches the current line, a false EOF response
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	49 (i.e. empty string) is returned instead. This lets the parser adhere to a
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	50 simple abstraction -- it parses until EOF closes the current message.
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	51 """
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	52 def __init__(self):
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	53 # The last partial line pushed into this object.
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	54 self._partial = ''
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	55 # The list of full, pushed lines, in reverse order
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	56 self._lines = []
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	57 # The stack of false-EOF checking predicates.
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	58 self._eofstack = []
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	59 # A flag indicating whether the file has been closed or not.
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	60 self._closed = False
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	61
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	62 def push_eof_matcher(self, pred):
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	63 self._eofstack.append(pred)
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	64
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	65 def pop_eof_matcher(self):
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	66 return self._eofstack.pop()
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	67
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	68 def close(self):
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	69 # Don't forget any trailing partial line.
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	70 self._lines.append(self._partial)
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	71 self._partial = ''
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	72 self._closed = True
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	73
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	74 def readline(self):
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	75 if not self._lines:
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	76 if self._closed:
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	77 return ''
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	78 return NeedMoreData
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	79 # Pop the line off the stack and see if it matches the current
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	80 # false-EOF predicate.
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	81 line = self._lines.pop()
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	82 # RFC 2046, section 5.1.2 requires us to recognize outer level
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	83 # boundaries at any level of inner nesting. Do this, but be sure it's
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	84 # in the order of most to least nested.
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	85 for ateof in self._eofstack[::-1]:
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	86 if ateof(line):
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	87 # We're at the false EOF. But push the last line back first.
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	88 self._lines.append(line)
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	89 return ''
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	90 return line
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	91
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	92 def unreadline(self, line):
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	93 # Let the consumer push a line back into the buffer.
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	94 assert line is not NeedMoreData
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	95 self._lines.append(line)
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	96
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	97 def push(self, data):
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	98 """Push some new data into this object."""
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	99 # Handle any previous leftovers
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	100 data, self._partial = self._partial + data, ''
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	101 # Crack into lines, but preserve the newlines on the end of each
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	102 parts = NLCRE_crack.split(data)
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	103 # The ahem interesting behaviour of re.split when supplied grouping
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	104 # parentheses is that the last element of the resulting list is the
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	105 # data after the final RE. In the case of a NL/CR terminated string,
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	106 # this is the empty string.
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	107 self._partial = parts.pop()
63047 3c73010bd0b0 #1555570: correctly handle a \r\n that is split by the read buffer. R. David Murray <rdmurray@bitdance.com> parents: 62347 diff changeset	108 #GAN 29Mar09 bugs 1555570, 1721862 Confusion at 8K boundary ending with \r:
3c73010bd0b0 #1555570: correctly handle a \r\n that is split by the read buffer. R. David Murray <rdmurray@bitdance.com> parents: 62347 diff changeset	109 # is there a \n to follow later?
3c73010bd0b0 #1555570: correctly handle a \r\n that is split by the read buffer. R. David Murray <rdmurray@bitdance.com> parents: 62347 diff changeset	110 if not self._partial and parts and parts[-1].endswith('\r'):
3c73010bd0b0 #1555570: correctly handle a \r\n that is split by the read buffer. R. David Murray <rdmurray@bitdance.com> parents: 62347 diff changeset	111 self._partial = parts.pop(-2)+parts.pop()
43019 d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	112 # parts is a list of strings, alternating between the line contents
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	113 # and the eol character(s). Gather up a list of lines after
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	114 # re-attaching the newlines.
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	115 lines = []
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	116 for i in range(len(parts) // 2):
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	117 lines.append(parts[i2] + parts[i2+1])
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	118 self.pushlines(lines)
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	119
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	120 def pushlines(self, lines):
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	121 # Reverse and insert at the front of the lines.
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	122 self._lines[:0] = lines[::-1]
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	123
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	124 def __iter__(self):
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	125 return self
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	126
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	127 def __next__(self):
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	128 line = self.readline()
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	129 if line == '':
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	130 raise StopIteration
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	131 return line
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	132
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	133
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	134
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	135 class FeedParser:
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	136 """A feed-style parser of email."""
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	137
77146 9388c671d52d #14731: refactor email policy framework. R David Murray <rdmurray@bitdance.com> parents: 75770 diff changeset	138 def __init__(self, _factory=message.Message, *, policy=compat32):
69428 2f3fda9d3906 #11731: simplify/enhance parser/generator API by introducing policy objects. R David Murray <rdmurray@bitdance.com> parents: 69377 diff changeset	139 """_factory is called with no arguments to create a new message obj
2f3fda9d3906 #11731: simplify/enhance parser/generator API by introducing policy objects. R David Murray <rdmurray@bitdance.com> parents: 69377 diff changeset	140
2f3fda9d3906 #11731: simplify/enhance parser/generator API by introducing policy objects. R David Murray <rdmurray@bitdance.com> parents: 69377 diff changeset	141 The policy keyword specifies a policy object that controls a number of
2f3fda9d3906 #11731: simplify/enhance parser/generator API by introducing policy objects. R David Murray <rdmurray@bitdance.com> parents: 69377 diff changeset	142 aspects of the parser's operation. The default policy maintains
2f3fda9d3906 #11731: simplify/enhance parser/generator API by introducing policy objects. R David Murray <rdmurray@bitdance.com> parents: 69377 diff changeset	143 backward compatibility.
2f3fda9d3906 #11731: simplify/enhance parser/generator API by introducing policy objects. R David Murray <rdmurray@bitdance.com> parents: 69377 diff changeset	144
2f3fda9d3906 #11731: simplify/enhance parser/generator API by introducing policy objects. R David Murray <rdmurray@bitdance.com> parents: 69377 diff changeset	145 """
43019 d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	146 self._factory = _factory
69428 2f3fda9d3906 #11731: simplify/enhance parser/generator API by introducing policy objects. R David Murray <rdmurray@bitdance.com> parents: 69377 diff changeset	147 self.policy = policy
77146 9388c671d52d #14731: refactor email policy framework. R David Murray <rdmurray@bitdance.com> parents: 75770 diff changeset	148 try:
9388c671d52d #14731: refactor email policy framework. R David Murray <rdmurray@bitdance.com> parents: 75770 diff changeset	149 _factory(policy=self.policy)
9388c671d52d #14731: refactor email policy framework. R David Murray <rdmurray@bitdance.com> parents: 75770 diff changeset	150 self._factory_kwds = lambda: {'policy': self.policy}
9388c671d52d #14731: refactor email policy framework. R David Murray <rdmurray@bitdance.com> parents: 75770 diff changeset	151 except TypeError:
9388c671d52d #14731: refactor email policy framework. R David Murray <rdmurray@bitdance.com> parents: 75770 diff changeset	152 # Assume this is an old-style factory
9388c671d52d #14731: refactor email policy framework. R David Murray <rdmurray@bitdance.com> parents: 75770 diff changeset	153 self._factory_kwds = lambda: {}
43019 d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	154 self._input = BufferedSubFile()
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	155 self._msgstack = []
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	156 self._parse = self._parsegen().__next__
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	157 self._cur = None
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	158 self._last = None
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	159 self._headersonly = False
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	160
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	161 # Non-public interface for supporting Parser's headersonly flag
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	162 def _set_headersonly(self):
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	163 self._headersonly = True
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	164
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	165 def feed(self, data):
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	166 """Push more data into the parser."""
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	167 self._input.push(data)
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	168 self._call_parse()
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	169
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	170 def _call_parse(self):
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	171 try:
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	172 self._parse()
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	173 except StopIteration:
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	174 pass
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	175
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	176 def close(self):
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	177 """Parse all remaining data and return the root message object."""
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	178 self._input.close()
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	179 self._call_parse()
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	180 root = self._pop_message()
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	181 assert not self._msgstack
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	182 # Look for final set of defects
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	183 if root.get_content_maintype() == 'multipart' \
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	184 and not root.is_multipart():
69428 2f3fda9d3906 #11731: simplify/enhance parser/generator API by introducing policy objects. R David Murray <rdmurray@bitdance.com> parents: 69377 diff changeset	185 defect = errors.MultipartInvariantViolationDefect()
2f3fda9d3906 #11731: simplify/enhance parser/generator API by introducing policy objects. R David Murray <rdmurray@bitdance.com> parents: 69377 diff changeset	186 self.policy.handle_defect(root, defect)
43019 d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	187 return root
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	188
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	189 def _new_message(self):
77146 9388c671d52d #14731: refactor email policy framework. R David Murray <rdmurray@bitdance.com> parents: 75770 diff changeset	190 msg = self._factory(**self._factory_kwds())
43019 d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	191 if self._cur and self._cur.get_content_type() == 'multipart/digest':
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	192 msg.set_default_type('message/rfc822')
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	193 if self._msgstack:
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	194 self._msgstack[-1].attach(msg)
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	195 self._msgstack.append(msg)
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	196 self._cur = msg
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	197 self._last = msg
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	198
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	199 def _pop_message(self):
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	200 retval = self._msgstack.pop()
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	201 if self._msgstack:
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	202 self._cur = self._msgstack[-1]
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	203 else:
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	204 self._cur = None
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	205 return retval
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	206
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	207 def _parsegen(self):
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	208 # Create a new message and start by parsing headers.
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	209 self._new_message()
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	210 headers = []
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	211 # Collect the headers, searching for a line that doesn't match the RFC
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	212 # 2822 header or continuation pattern (including an empty line).
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	213 for line in self._input:
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	214 if line is NeedMoreData:
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	215 yield NeedMoreData
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	216 continue
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	217 if not headerRE.match(line):
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	218 # If we saw the RFC defined header/body separator
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	219 # (i.e. newline), just throw it away. Otherwise the line is
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	220 # part of the body so push it back.
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	221 if not NLCRE.match(line):
77191 0869f5f47608 #14925: email now registers a defect for missing header/body separator. R David Murray <rdmurray@bitdance.com> parents: 77146 diff changeset	222 defect = errors.MissingHeaderBodySeparatorDefect()
0869f5f47608 #14925: email now registers a defect for missing header/body separator. R David Murray <rdmurray@bitdance.com> parents: 77146 diff changeset	223 self.policy.handle_defect(self._cur, defect)
43019 d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	224 self._input.unreadline(line)
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	225 break
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	226 headers.append(line)
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	227 # Done with the headers, so parse them and figure out what we're
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	228 # supposed to see in the body of the message.
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	229 self._parse_headers(headers)
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	230 # Headers-only parsing is a backwards compatibility hack, which was
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	231 # necessary in the older parser, which could throw errors. All
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	232 # remaining lines in the input are thrown into the message body.
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	233 if self._headersonly:
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	234 lines = []
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	235 while True:
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	236 line = self._input.readline()
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	237 if line is NeedMoreData:
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	238 yield NeedMoreData
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	239 continue
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	240 if line == '':
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	241 break
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	242 lines.append(line)
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	243 self._cur.set_payload(EMPTYSTRING.join(lines))
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	244 return
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	245 if self._cur.get_content_type() == 'message/delivery-status':
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	246 # message/delivery-status contains blocks of headers separated by
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	247 # a blank line. We'll represent each header block as a separate
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	248 # nested message object, but the processing is a bit different
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	249 # than standard message/* types because there is no body for the
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	250 # nested messages. A blank line separates the subparts.
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	251 while True:
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	252 self._input.push_eof_matcher(NLCRE.match)
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	253 for retval in self._parsegen():
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	254 if retval is NeedMoreData:
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	255 yield NeedMoreData
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	256 continue
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	257 break
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	258 msg = self._pop_message()
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	259 # We need to pop the EOF matcher in order to tell if we're at
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	260 # the end of the current file, not the end of the last block
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	261 # of message headers.
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	262 self._input.pop_eof_matcher()
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	263 # The input stream must be sitting at the newline or at the
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	264 # EOF. We want to see if we're at the end of this subpart, so
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	265 # first consume the blank line, then test the next line to see
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	266 # if we're at this subpart's EOF.
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	267 while True:
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	268 line = self._input.readline()
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	269 if line is NeedMoreData:
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	270 yield NeedMoreData
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	271 continue
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	272 break
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	273 while True:
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	274 line = self._input.readline()
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	275 if line is NeedMoreData:
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	276 yield NeedMoreData
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	277 continue
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	278 break
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	279 if line == '':
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	280 break
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	281 # Not at EOF so this is a line we're going to need.
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	282 self._input.unreadline(line)
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	283 return
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	284 if self._cur.get_content_maintype() == 'message':
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	285 # The message claims to be a message/* type, then what follows is
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	286 # another RFC 2822 message.
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	287 for retval in self._parsegen():
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	288 if retval is NeedMoreData:
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	289 yield NeedMoreData
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	290 continue
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	291 break
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	292 self._pop_message()
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	293 return
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	294 if self._cur.get_content_maintype() == 'multipart':
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	295 boundary = self._cur.get_boundary()
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	296 if boundary is None:
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	297 # The message /claims/ to be a multipart but it has not
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	298 # defined a boundary. That's a problem which we'll handle by
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	299 # reading everything until the EOF and marking the message as
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	300 # defective.
69428 2f3fda9d3906 #11731: simplify/enhance parser/generator API by introducing policy objects. R David Murray <rdmurray@bitdance.com> parents: 69377 diff changeset	301 defect = errors.NoBoundaryInMultipartDefect()
2f3fda9d3906 #11731: simplify/enhance parser/generator API by introducing policy objects. R David Murray <rdmurray@bitdance.com> parents: 69377 diff changeset	302 self.policy.handle_defect(self._cur, defect)
43019 d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	303 lines = []
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	304 for line in self._input:
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	305 if line is NeedMoreData:
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	306 yield NeedMoreData
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	307 continue
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	308 lines.append(line)
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	309 self._cur.set_payload(EMPTYSTRING.join(lines))
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	310 return
70925 5a2602939d5d #1874: detect invalid multipart CTE and report it as a defect. R David Murray <rdmurray@bitdance.com> parents: 69428 diff changeset	311 # Make sure a valid content type was specified per RFC 2045:6.4.
5a2602939d5d #1874: detect invalid multipart CTE and report it as a defect. R David Murray <rdmurray@bitdance.com> parents: 69428 diff changeset	312 if (self._cur.get('content-transfer-encoding', '8bit').lower()
5a2602939d5d #1874: detect invalid multipart CTE and report it as a defect. R David Murray <rdmurray@bitdance.com> parents: 69428 diff changeset	313 not in ('7bit', '8bit', 'binary')):
5a2602939d5d #1874: detect invalid multipart CTE and report it as a defect. R David Murray <rdmurray@bitdance.com> parents: 69428 diff changeset	314 defect = errors.InvalidMultipartContentTransferEncodingDefect()
5a2602939d5d #1874: detect invalid multipart CTE and report it as a defect. R David Murray <rdmurray@bitdance.com> parents: 69428 diff changeset	315 self.policy.handle_defect(self._cur, defect)
43019 d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	316 # Create a line match predicate which matches the inter-part
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	317 # boundary as well as the end-of-multipart boundary. Don't push
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	318 # this onto the input stream until we've scanned past the
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	319 # preamble.
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	320 separator = '--' + boundary
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	321 boundaryre = re.compile(
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	322 '(?P<sep>' + re.escape(separator) +
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	323 r')(?P<end>--)?(?P<ws>[ \t]*)(?P<linesep>\r\n\|\r\|\n)?$')
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	324 capturing_preamble = True
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	325 preamble = []
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	326 linesep = False
77198 81e008f13b4f #12515: email now registers a defect if the MIME end boundary is missing. R David Murray <rdmurray@bitdance.com> parents: 77191 diff changeset	327 close_boundary_seen = False
43019 d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	328 while True:
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	329 line = self._input.readline()
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	330 if line is NeedMoreData:
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	331 yield NeedMoreData
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	332 continue
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	333 if line == '':
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	334 break
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	335 mo = boundaryre.match(line)
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	336 if mo:
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	337 # If we're looking at the end boundary, we're done with
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	338 # this multipart. If there was a newline at the end of
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	339 # the closing boundary, then we need to initialize the
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	340 # epilogue with the empty string (see below).
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	341 if mo.group('end'):
77198 81e008f13b4f #12515: email now registers a defect if the MIME end boundary is missing. R David Murray <rdmurray@bitdance.com> parents: 77191 diff changeset	342 close_boundary_seen = True
43019 d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	343 linesep = mo.group('linesep')
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	344 break
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	345 # We saw an inter-part boundary. Were we in the preamble?
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	346 if capturing_preamble:
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	347 if preamble:
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	348 # According to RFC 2046, the last newline belongs
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	349 # to the boundary.
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	350 lastline = preamble[-1]
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	351 eolmo = NLCRE_eol.search(lastline)
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	352 if eolmo:
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	353 preamble[-1] = lastline[:-len(eolmo.group(0))]
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	354 self._cur.preamble = EMPTYSTRING.join(preamble)
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	355 capturing_preamble = False
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	356 self._input.unreadline(line)
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	357 continue
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	358 # We saw a boundary separating two parts. Consume any
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	359 # multiple boundary lines that may be following. Our
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	360 # interpretation of RFC 2046 BNF grammar does not produce
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	361 # body parts within such double boundaries.
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	362 while True:
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	363 line = self._input.readline()
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	364 if line is NeedMoreData:
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	365 yield NeedMoreData
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	366 continue
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	367 mo = boundaryre.match(line)
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	368 if not mo:
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	369 self._input.unreadline(line)
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	370 break
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	371 # Recurse to parse this subpart; the input stream points
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	372 # at the subpart's first line.
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	373 self._input.push_eof_matcher(boundaryre.match)
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	374 for retval in self._parsegen():
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	375 if retval is NeedMoreData:
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	376 yield NeedMoreData
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	377 continue
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	378 break
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	379 # Because of RFC 2046, the newline preceding the boundary
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	380 # separator actually belongs to the boundary, not the
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	381 # previous subpart's payload (or epilogue if the previous
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	382 # part is a multipart).
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	383 if self._last.get_content_maintype() == 'multipart':
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	384 epilogue = self._last.epilogue
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	385 if epilogue == '':
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	386 self._last.epilogue = None
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	387 elif epilogue is not None:
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	388 mo = NLCRE_eol.search(epilogue)
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	389 if mo:
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	390 end = len(mo.group(0))
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	391 self._last.epilogue = epilogue[:-end]
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	392 else:
69171 b807cf929e26 #11605: don't use set/get_payload in feedparser; they do conversions. R David Murray <rdmurray@bitdance.com> parents: 65180 diff changeset	393 payload = self._last._payload
43577 ace45b361e87 Patch# 1258 by Christian Heimes: kill basestring. Guido van Rossum <guido@python.org> parents: 43019 diff changeset	394 if isinstance(payload, str):
43019 d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	395 mo = NLCRE_eol.search(payload)
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	396 if mo:
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	397 payload = payload[:-len(mo.group(0))]
69171 b807cf929e26 #11605: don't use set/get_payload in feedparser; they do conversions. R David Murray <rdmurray@bitdance.com> parents: 65180 diff changeset	398 self._last._payload = payload
43019 d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	399 self._input.pop_eof_matcher()
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	400 self._pop_message()
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	401 # Set the multipart up for newline cleansing, which will
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	402 # happen if we're in a nested multipart.
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	403 self._last = self._cur
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	404 else:
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	405 # I think we must be in the preamble
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	406 assert capturing_preamble
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	407 preamble.append(line)
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	408 # We've seen either the EOF or the end boundary. If we're still
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	409 # capturing the preamble, we never saw the start boundary. Note
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	410 # that as a defect and store the captured text as the payload.
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	411 if capturing_preamble:
69428 2f3fda9d3906 #11731: simplify/enhance parser/generator API by introducing policy objects. R David Murray <rdmurray@bitdance.com> parents: 69377 diff changeset	412 defect = errors.StartBoundaryNotFoundDefect()
2f3fda9d3906 #11731: simplify/enhance parser/generator API by introducing policy objects. R David Murray <rdmurray@bitdance.com> parents: 69377 diff changeset	413 self.policy.handle_defect(self._cur, defect)
43019 d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	414 self._cur.set_payload(EMPTYSTRING.join(preamble))
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	415 epilogue = []
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	416 for line in self._input:
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	417 if line is NeedMoreData:
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	418 yield NeedMoreData
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	419 continue
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	420 self._cur.epilogue = EMPTYSTRING.join(epilogue)
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	421 return
77198 81e008f13b4f #12515: email now registers a defect if the MIME end boundary is missing. R David Murray <rdmurray@bitdance.com> parents: 77191 diff changeset	422 # If we're not processing the preamble, then we might have seen
81e008f13b4f #12515: email now registers a defect if the MIME end boundary is missing. R David Murray <rdmurray@bitdance.com> parents: 77191 diff changeset	423 # EOF without seeing that end boundary...that is also a defect.
81e008f13b4f #12515: email now registers a defect if the MIME end boundary is missing. R David Murray <rdmurray@bitdance.com> parents: 77191 diff changeset	424 if not close_boundary_seen:
81e008f13b4f #12515: email now registers a defect if the MIME end boundary is missing. R David Murray <rdmurray@bitdance.com> parents: 77191 diff changeset	425 defect = errors.CloseBoundaryNotFoundDefect()
81e008f13b4f #12515: email now registers a defect if the MIME end boundary is missing. R David Murray <rdmurray@bitdance.com> parents: 77191 diff changeset	426 self.policy.handle_defect(self._cur, defect)
81e008f13b4f #12515: email now registers a defect if the MIME end boundary is missing. R David Murray <rdmurray@bitdance.com> parents: 77191 diff changeset	427 return
81e008f13b4f #12515: email now registers a defect if the MIME end boundary is missing. R David Murray <rdmurray@bitdance.com> parents: 77191 diff changeset	428 # Everything from here to the EOF is epilogue. If the end boundary
81e008f13b4f #12515: email now registers a defect if the MIME end boundary is missing. R David Murray <rdmurray@bitdance.com> parents: 77191 diff changeset	429 # ended in a newline, we'll need to make sure the epilogue isn't
81e008f13b4f #12515: email now registers a defect if the MIME end boundary is missing. R David Murray <rdmurray@bitdance.com> parents: 77191 diff changeset	430 # None
43019 d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	431 if linesep:
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	432 epilogue = ['']
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	433 else:
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	434 epilogue = []
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	435 for line in self._input:
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	436 if line is NeedMoreData:
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	437 yield NeedMoreData
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	438 continue
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	439 epilogue.append(line)
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	440 # Any CRLF at the front of the epilogue is not technically part of
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	441 # the epilogue. Also, watch out for an empty string epilogue,
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	442 # which means a single newline.
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	443 if epilogue:
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	444 firstline = epilogue[0]
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	445 bolmo = NLCRE_bol.match(firstline)
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	446 if bolmo:
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	447 epilogue[0] = firstline[len(bolmo.group(0)):]
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	448 self._cur.epilogue = EMPTYSTRING.join(epilogue)
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	449 return
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	450 # Otherwise, it's some non-multipart type, so the entire rest of the
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	451 # file contents becomes the payload.
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	452 lines = []
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	453 for line in self._input:
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	454 if line is NeedMoreData:
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	455 yield NeedMoreData
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	456 continue
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	457 lines.append(line)
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	458 self._cur.set_payload(EMPTYSTRING.join(lines))
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	459
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	460 def _parse_headers(self, lines):
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	461 # Passed a list of lines that make up the headers for the current msg
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	462 lastheader = ''
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	463 lastvalue = []
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	464 for lineno, line in enumerate(lines):
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	465 # Check for continuation
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	466 if line[0] in ' \t':
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	467 if not lastheader:
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	468 # The first line of the headers was a continuation. This
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	469 # is illegal, so let's note the defect, store the illegal
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	470 # line, and ignore it for purposes of headers.
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	471 defect = errors.FirstHeaderLineIsContinuationDefect(line)
69428 2f3fda9d3906 #11731: simplify/enhance parser/generator API by introducing policy objects. R David Murray <rdmurray@bitdance.com> parents: 69377 diff changeset	472 self.policy.handle_defect(self._cur, defect)
43019 d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	473 continue
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	474 lastvalue.append(line)
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	475 continue
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	476 if lastheader:
77146 9388c671d52d #14731: refactor email policy framework. R David Murray <rdmurray@bitdance.com> parents: 75770 diff changeset	477 self._cur.set_raw(*self.policy.header_source_parse(lastvalue))
43019 d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	478 lastheader, lastvalue = '', []
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	479 # Check for envelope header, i.e. unix-from
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	480 if line.startswith('From '):
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	481 if lineno == 0:
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	482 # Strip off the trailing newline
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	483 mo = NLCRE_eol.search(line)
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	484 if mo:
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	485 line = line[:-len(mo.group(0))]
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	486 self._cur.set_unixfrom(line)
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	487 continue
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	488 elif lineno == len(lines) - 1:
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	489 # Something looking like a unix-from at the end - it's
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	490 # probably the first line of the body, so push back the
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	491 # line and stop.
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	492 self._input.unreadline(line)
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	493 return
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	494 else:
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	495 # Weirdly placed unix-from line. Note this as a defect
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	496 # and ignore it.
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	497 defect = errors.MisplacedEnvelopeHeaderDefect(line)
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	498 self._cur.defects.append(defect)
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	499 continue
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	500 # Split the line on the colon separating field name from value.
77191 0869f5f47608 #14925: email now registers a defect for missing header/body separator. R David Murray <rdmurray@bitdance.com> parents: 77146 diff changeset	501 # There will always be a colon, because if there wasn't the part of
0869f5f47608 #14925: email now registers a defect for missing header/body separator. R David Murray <rdmurray@bitdance.com> parents: 77146 diff changeset	502 # the parser that calls us would have started parsing the body.
43019 d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	503 i = line.find(':')
77191 0869f5f47608 #14925: email now registers a defect for missing header/body separator. R David Murray <rdmurray@bitdance.com> parents: 77146 diff changeset	504 assert i>0, "_parse_headers fed line with no : and no leading WS"
43019 d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	505 lastheader = line[:i]
77146 9388c671d52d #14731: refactor email policy framework. R David Murray <rdmurray@bitdance.com> parents: 75770 diff changeset	506 lastvalue = [line]
43019 d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	507 # Done with all the lines, so handle the last header.
d5f3c2f416f2 Copying the email package back, despite its failings. Guido van Rossum <guido@python.org> parents: diff changeset	508 if lastheader:
77146 9388c671d52d #14731: refactor email policy framework. R David Murray <rdmurray@bitdance.com> parents: 75770 diff changeset	509 self._cur.set_raw(*self.policy.header_source_parse(lastvalue))
65180 1a041f364916 #4661: add bytes parsing and generation to email (email version bump to 5.1.0) R. David Murray <rdmurray@bitdance.com> parents: 63047 diff changeset	510
77146 9388c671d52d #14731: refactor email policy framework. R David Murray <rdmurray@bitdance.com> parents: 75770 diff changeset	511
65180 1a041f364916 #4661: add bytes parsing and generation to email (email version bump to 5.1.0) R. David Murray <rdmurray@bitdance.com> parents: 63047 diff changeset	512 class BytesFeedParser(FeedParser):
1a041f364916 #4661: add bytes parsing and generation to email (email version bump to 5.1.0) R. David Murray <rdmurray@bitdance.com> parents: 63047 diff changeset	513 """Like FeedParser, but feed accepts bytes."""
1a041f364916 #4661: add bytes parsing and generation to email (email version bump to 5.1.0) R. David Murray <rdmurray@bitdance.com> parents: 63047 diff changeset	514
1a041f364916 #4661: add bytes parsing and generation to email (email version bump to 5.1.0) R. David Murray <rdmurray@bitdance.com> parents: 63047 diff changeset	515 def feed(self, data):
1a041f364916 #4661: add bytes parsing and generation to email (email version bump to 5.1.0) R. David Murray <rdmurray@bitdance.com> parents: 63047 diff changeset	516 super().feed(data.decode('ascii', 'surrogateescape'))

Mercurial > cpython-lru_cache-weakref

annotate Lib/email/feedparser.py @ 79691:05c8928362ee