cpython-lru_cache-weakref: Lib/HTMLParser.py annotate

annotate Lib/HTMLParser.py @ 24042:63fbbed10714 legacy-trunk

Simplify HTTPSConnection constructor. See discussion in SF bug 458463.

author	Jeremy Hylton <jeremy@alum.mit.edu>
date	Fri, 28 Jun 2002 23:38:14 +0000
parents	6bae0c31bbfb
children	2c22b5c09a58

rev	line source
18984 00935031b373 Change some comments into docstrings. Fred Drake <fdrake@acm.org> parents: 17934 diff changeset	1 """A parser for HTML and XHTML."""
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	2
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	3 # This file is based on sgmllib.py, but the API is slightly different.
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	4
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	5 # XXX There should be a way to distinguish between PCDATA (parsed
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	6 # character data -- the normal case), RCDATA (replaceable character
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	7 # data -- only char and entity references and end tags are special)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	8 # and CDATA (character data -- only end tags are special).
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	9
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	10
20067 299f5fcb0ba8 Re-factor the HTMLParser class to use the new markupbase.ParserBase class. Fred Drake <fdrake@acm.org> parents: 19953 diff changeset	11 import markupbase
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	12 import re
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	13
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	14 # Regular expressions used for parsing
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	15
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	16 interesting_normal = re.compile('[&<]')
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	17 interesting_cdata = re.compile(r'<(/\|\Z)')
19682 8dfb13a4e175 Added reasonable parsing of the DOCTYPE declaration, fixed edge cases Fred Drake <fdrake@acm.org> parents: 19410 diff changeset	18 incomplete = re.compile('&[a-zA-Z#]')
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	19
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	20 entityref = re.compile('&([a-zA-Z][-.a-zA-Z0-9]*)[^a-zA-Z0-9]')
18984 00935031b373 Change some comments into docstrings. Fred Drake <fdrake@acm.org> parents: 17934 diff changeset	21 charref = re.compile('&#(?:[0-9]+\|[xX][0-9a-fA-F]+)[^0-9a-fA-F]')
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	22
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	23 starttagopen = re.compile('<[a-zA-Z]')
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	24 piclose = re.compile('>')
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	25 commentclose = re.compile(r'--\s*>')
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	26 tagfind = re.compile('[a-zA-Z][-.a-zA-Z0-9:_]*')
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	27 attrfind = re.compile(
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	28 r'\s([a-zA-Z_][-.:a-zA-Z_0-9])(\s=\s'
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	29 r'(\'[^\']\'\|"[^"]"\|[-a-zA-Z0-9./:;+%?!&$_#=~]))?')
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	30
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	31 locatestarttagend = re.compile(r"""
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	32 <[a-zA-Z][-.a-zA-Z0-9:_]* # tag name
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	33 (?:\s+ # whitespace before attribute name
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	34 (?:[a-zA-Z_][-.:a-zA-Z0-9_]* # attribute name
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	35 (?:\s=\s # value indicator
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	36 (?:'[^']*' # LITA-enclosed value
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	37 \|\"[^\"]*\" # LIT-enclosed value
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	38 \|[^'\">\s]+ # bare value
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	39 )
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	40 )?
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	41 )
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	42 )*
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	43 \s* # trailing whitespace
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	44 """, re.VERBOSE)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	45 endendtag = re.compile('>')
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	46 endtagfind = re.compile('</\s([a-zA-Z][-.a-zA-Z0-9:_])\s*>')
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	47
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	48
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	49 class HTMLParseError(Exception):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	50 """Exception raised for all parse errors."""
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	51
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	52 def __init__(self, msg, position=(None, None)):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	53 assert msg
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	54 self.msg = msg
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	55 self.lineno = position[0]
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	56 self.offset = position[1]
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	57
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	58 def __str__(self):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	59 result = self.msg
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	60 if self.lineno is not None:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	61 result = result + ", at line %d" % self.lineno
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	62 if self.offset is not None:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	63 result = result + ", column %d" % (self.offset + 1)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	64 return result
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	65
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	66
20067 299f5fcb0ba8 Re-factor the HTMLParser class to use the new markupbase.ParserBase class. Fred Drake <fdrake@acm.org> parents: 19953 diff changeset	67 class HTMLParser(markupbase.ParserBase):
18984 00935031b373 Change some comments into docstrings. Fred Drake <fdrake@acm.org> parents: 17934 diff changeset	68 """Find tags and other markup and call handler functions.
00935031b373 Change some comments into docstrings. Fred Drake <fdrake@acm.org> parents: 17934 diff changeset	69
00935031b373 Change some comments into docstrings. Fred Drake <fdrake@acm.org> parents: 17934 diff changeset	70 Usage:
00935031b373 Change some comments into docstrings. Fred Drake <fdrake@acm.org> parents: 17934 diff changeset	71 p = HTMLParser()
00935031b373 Change some comments into docstrings. Fred Drake <fdrake@acm.org> parents: 17934 diff changeset	72 p.feed(data)
00935031b373 Change some comments into docstrings. Fred Drake <fdrake@acm.org> parents: 17934 diff changeset	73 ...
00935031b373 Change some comments into docstrings. Fred Drake <fdrake@acm.org> parents: 17934 diff changeset	74 p.close()
17932 ce7f0e380830 Removed incorrect comment left over from sgmllib.py. Guido van Rossum <guido@python.org> parents: 17853 diff changeset	75
18984 00935031b373 Change some comments into docstrings. Fred Drake <fdrake@acm.org> parents: 17934 diff changeset	76 Start tags are handled by calling self.handle_starttag() or
00935031b373 Change some comments into docstrings. Fred Drake <fdrake@acm.org> parents: 17934 diff changeset	77 self.handle_startendtag(); end tags by self.handle_endtag(). The
00935031b373 Change some comments into docstrings. Fred Drake <fdrake@acm.org> parents: 17934 diff changeset	78 data between tags is passed from the parser to the derived class
00935031b373 Change some comments into docstrings. Fred Drake <fdrake@acm.org> parents: 17934 diff changeset	79 by calling self.handle_data() with the data as argument (the data
00935031b373 Change some comments into docstrings. Fred Drake <fdrake@acm.org> parents: 17934 diff changeset	80 may be split up in arbitrary chunks). Entity references are
00935031b373 Change some comments into docstrings. Fred Drake <fdrake@acm.org> parents: 17934 diff changeset	81 passed by calling self.handle_entityref() with the entity
00935031b373 Change some comments into docstrings. Fred Drake <fdrake@acm.org> parents: 17934 diff changeset	82 reference as the argument. Numeric character references are
00935031b373 Change some comments into docstrings. Fred Drake <fdrake@acm.org> parents: 17934 diff changeset	83 passed to self.handle_charref() with the string containing the
00935031b373 Change some comments into docstrings. Fred Drake <fdrake@acm.org> parents: 17934 diff changeset	84 reference as the argument.
00935031b373 Change some comments into docstrings. Fred Drake <fdrake@acm.org> parents: 17934 diff changeset	85 """
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	86
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	87 CDATA_CONTENT_ELEMENTS = ("script", "style")
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	88
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	89
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	90 def __init__(self):
18984 00935031b373 Change some comments into docstrings. Fred Drake <fdrake@acm.org> parents: 17934 diff changeset	91 """Initialize and reset this instance."""
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	92 self.reset()
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	93
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	94 def reset(self):
18984 00935031b373 Change some comments into docstrings. Fred Drake <fdrake@acm.org> parents: 17934 diff changeset	95 """Reset this instance. Loses all unprocessed data."""
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	96 self.rawdata = ''
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	97 self.lasttag = '???'
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	98 self.interesting = interesting_normal
20067 299f5fcb0ba8 Re-factor the HTMLParser class to use the new markupbase.ParserBase class. Fred Drake <fdrake@acm.org> parents: 19953 diff changeset	99 markupbase.ParserBase.reset(self)
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	100
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	101 def feed(self, data):
18984 00935031b373 Change some comments into docstrings. Fred Drake <fdrake@acm.org> parents: 17934 diff changeset	102 """Feed data to the parser.
00935031b373 Change some comments into docstrings. Fred Drake <fdrake@acm.org> parents: 17934 diff changeset	103
00935031b373 Change some comments into docstrings. Fred Drake <fdrake@acm.org> parents: 17934 diff changeset	104 Call this as often as you want, with as little or as much text
00935031b373 Change some comments into docstrings. Fred Drake <fdrake@acm.org> parents: 17934 diff changeset	105 as you want (may include '\n').
00935031b373 Change some comments into docstrings. Fred Drake <fdrake@acm.org> parents: 17934 diff changeset	106 """
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	107 self.rawdata = self.rawdata + data
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	108 self.goahead(0)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	109
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	110 def close(self):
18984 00935031b373 Change some comments into docstrings. Fred Drake <fdrake@acm.org> parents: 17934 diff changeset	111 """Handle any buffered data."""
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	112 self.goahead(1)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	113
20067 299f5fcb0ba8 Re-factor the HTMLParser class to use the new markupbase.ParserBase class. Fred Drake <fdrake@acm.org> parents: 19953 diff changeset	114 def error(self, message):
299f5fcb0ba8 Re-factor the HTMLParser class to use the new markupbase.ParserBase class. Fred Drake <fdrake@acm.org> parents: 19953 diff changeset	115 raise HTMLParseError(message, self.getpos())
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	116
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	117 __starttag_text = None
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	118
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	119 def get_starttag_text(self):
18984 00935031b373 Change some comments into docstrings. Fred Drake <fdrake@acm.org> parents: 17934 diff changeset	120 """Return full source of start tag: '<...>'."""
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	121 return self.__starttag_text
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	122
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	123 def set_cdata_mode(self):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	124 self.interesting = interesting_cdata
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	125
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	126 def clear_cdata_mode(self):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	127 self.interesting = interesting_normal
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	128
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	129 # Internal -- handle data as far as reasonable. May leave state
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	130 # and data to be processed by a subsequent call. If 'end' is
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	131 # true, force handling all data as if followed by EOF marker.
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	132 def goahead(self, end):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	133 rawdata = self.rawdata
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	134 i = 0
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	135 n = len(rawdata)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	136 while i < n:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	137 match = self.interesting.search(rawdata, i) # < or &
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	138 if match:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	139 j = match.start()
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	140 else:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	141 j = n
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	142 if i < j: self.handle_data(rawdata[i:j])
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	143 i = self.updatepos(i, j)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	144 if i == n: break
21158 1acc4ba73b9d Convert to using string methods instead of the string module. Fred Drake <fdrake@acm.org> parents: 20067 diff changeset	145 startswith = rawdata.startswith
1acc4ba73b9d Convert to using string methods instead of the string module. Fred Drake <fdrake@acm.org> parents: 20067 diff changeset	146 if startswith('<', i):
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	147 if starttagopen.match(rawdata, i): # < + letter
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	148 k = self.parse_starttag(i)
21158 1acc4ba73b9d Convert to using string methods instead of the string module. Fred Drake <fdrake@acm.org> parents: 20067 diff changeset	149 elif startswith("</", i):
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	150 k = self.parse_endtag(i)
21158 1acc4ba73b9d Convert to using string methods instead of the string module. Fred Drake <fdrake@acm.org> parents: 20067 diff changeset	151 elif startswith("<!--", i):
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	152 k = self.parse_comment(i)
21158 1acc4ba73b9d Convert to using string methods instead of the string module. Fred Drake <fdrake@acm.org> parents: 20067 diff changeset	153 elif startswith("<?", i):
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	154 k = self.parse_pi(i)
21158 1acc4ba73b9d Convert to using string methods instead of the string module. Fred Drake <fdrake@acm.org> parents: 20067 diff changeset	155 elif startswith("<!", i):
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	156 k = self.parse_declaration(i)
19682 8dfb13a4e175 Added reasonable parsing of the DOCTYPE declaration, fixed edge cases Fred Drake <fdrake@acm.org> parents: 19410 diff changeset	157 elif (i + 1) < n:
19410 c403af003884 Deal more appropriately with bare ampersands and pointy brackets; this Fred Drake <fdrake@acm.org> parents: 18984 diff changeset	158 self.handle_data("<")
c403af003884 Deal more appropriately with bare ampersands and pointy brackets; this Fred Drake <fdrake@acm.org> parents: 18984 diff changeset	159 k = i + 1
19682 8dfb13a4e175 Added reasonable parsing of the DOCTYPE declaration, fixed edge cases Fred Drake <fdrake@acm.org> parents: 19410 diff changeset	160 else:
8dfb13a4e175 Added reasonable parsing of the DOCTYPE declaration, fixed edge cases Fred Drake <fdrake@acm.org> parents: 19410 diff changeset	161 break
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	162 if k < 0:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	163 if end:
20067 299f5fcb0ba8 Re-factor the HTMLParser class to use the new markupbase.ParserBase class. Fred Drake <fdrake@acm.org> parents: 19953 diff changeset	164 self.error("EOF in middle of construct")
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	165 break
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	166 i = self.updatepos(i, k)
21158 1acc4ba73b9d Convert to using string methods instead of the string module. Fred Drake <fdrake@acm.org> parents: 20067 diff changeset	167 elif startswith("&#", i):
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	168 match = charref.match(rawdata, i)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	169 if match:
18984 00935031b373 Change some comments into docstrings. Fred Drake <fdrake@acm.org> parents: 17934 diff changeset	170 name = match.group()[2:-1]
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	171 self.handle_charref(name)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	172 k = match.end()
21158 1acc4ba73b9d Convert to using string methods instead of the string module. Fred Drake <fdrake@acm.org> parents: 20067 diff changeset	173 if not startswith(';', k-1):
19410 c403af003884 Deal more appropriately with bare ampersands and pointy brackets; this Fred Drake <fdrake@acm.org> parents: 18984 diff changeset	174 k = k - 1
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	175 i = self.updatepos(i, k)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	176 continue
19682 8dfb13a4e175 Added reasonable parsing of the DOCTYPE declaration, fixed edge cases Fred Drake <fdrake@acm.org> parents: 19410 diff changeset	177 else:
8dfb13a4e175 Added reasonable parsing of the DOCTYPE declaration, fixed edge cases Fred Drake <fdrake@acm.org> parents: 19410 diff changeset	178 break
21158 1acc4ba73b9d Convert to using string methods instead of the string module. Fred Drake <fdrake@acm.org> parents: 20067 diff changeset	179 elif startswith('&', i):
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	180 match = entityref.match(rawdata, i)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	181 if match:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	182 name = match.group(1)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	183 self.handle_entityref(name)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	184 k = match.end()
21158 1acc4ba73b9d Convert to using string methods instead of the string module. Fred Drake <fdrake@acm.org> parents: 20067 diff changeset	185 if not startswith(';', k-1):
19410 c403af003884 Deal more appropriately with bare ampersands and pointy brackets; this Fred Drake <fdrake@acm.org> parents: 18984 diff changeset	186 k = k - 1
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	187 i = self.updatepos(i, k)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	188 continue
19410 c403af003884 Deal more appropriately with bare ampersands and pointy brackets; this Fred Drake <fdrake@acm.org> parents: 18984 diff changeset	189 match = incomplete.match(rawdata, i)
c403af003884 Deal more appropriately with bare ampersands and pointy brackets; this Fred Drake <fdrake@acm.org> parents: 18984 diff changeset	190 if match:
19682 8dfb13a4e175 Added reasonable parsing of the DOCTYPE declaration, fixed edge cases Fred Drake <fdrake@acm.org> parents: 19410 diff changeset	191 # match.group() will contain at least 2 chars
21158 1acc4ba73b9d Convert to using string methods instead of the string module. Fred Drake <fdrake@acm.org> parents: 20067 diff changeset	192 if end and match.group() == rawdata[i:]:
20067 299f5fcb0ba8 Re-factor the HTMLParser class to use the new markupbase.ParserBase class. Fred Drake <fdrake@acm.org> parents: 19953 diff changeset	193 self.error("EOF in middle of entity or char ref")
19682 8dfb13a4e175 Added reasonable parsing of the DOCTYPE declaration, fixed edge cases Fred Drake <fdrake@acm.org> parents: 19410 diff changeset	194 # incomplete
8dfb13a4e175 Added reasonable parsing of the DOCTYPE declaration, fixed edge cases Fred Drake <fdrake@acm.org> parents: 19410 diff changeset	195 break
8dfb13a4e175 Added reasonable parsing of the DOCTYPE declaration, fixed edge cases Fred Drake <fdrake@acm.org> parents: 19410 diff changeset	196 elif (i + 1) < n:
8dfb13a4e175 Added reasonable parsing of the DOCTYPE declaration, fixed edge cases Fred Drake <fdrake@acm.org> parents: 19410 diff changeset	197 # not the end of the buffer, and can't be confused
8dfb13a4e175 Added reasonable parsing of the DOCTYPE declaration, fixed edge cases Fred Drake <fdrake@acm.org> parents: 19410 diff changeset	198 # with some other construct
8dfb13a4e175 Added reasonable parsing of the DOCTYPE declaration, fixed edge cases Fred Drake <fdrake@acm.org> parents: 19410 diff changeset	199 self.handle_data("&")
8dfb13a4e175 Added reasonable parsing of the DOCTYPE declaration, fixed edge cases Fred Drake <fdrake@acm.org> parents: 19410 diff changeset	200 i = self.updatepos(i, i + 1)
8dfb13a4e175 Added reasonable parsing of the DOCTYPE declaration, fixed edge cases Fred Drake <fdrake@acm.org> parents: 19410 diff changeset	201 else:
8dfb13a4e175 Added reasonable parsing of the DOCTYPE declaration, fixed edge cases Fred Drake <fdrake@acm.org> parents: 19410 diff changeset	202 break
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	203 else:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	204 assert 0, "interesting.search() lied"
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	205 # end while
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	206 if end and i < n:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	207 self.handle_data(rawdata[i:n])
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	208 i = self.updatepos(i, n)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	209 self.rawdata = rawdata[i:]
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	210
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	211 # Internal -- parse comment, return end or -1 if not terminated
19682 8dfb13a4e175 Added reasonable parsing of the DOCTYPE declaration, fixed edge cases Fred Drake <fdrake@acm.org> parents: 19410 diff changeset	212 def parse_comment(self, i, report=1):
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	213 rawdata = self.rawdata
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	214 assert rawdata[i:i+4] == '<!--', 'unexpected call to parse_comment()'
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	215 match = commentclose.search(rawdata, i+4)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	216 if not match:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	217 return -1
19682 8dfb13a4e175 Added reasonable parsing of the DOCTYPE declaration, fixed edge cases Fred Drake <fdrake@acm.org> parents: 19410 diff changeset	218 if report:
8dfb13a4e175 Added reasonable parsing of the DOCTYPE declaration, fixed edge cases Fred Drake <fdrake@acm.org> parents: 19410 diff changeset	219 j = match.start()
8dfb13a4e175 Added reasonable parsing of the DOCTYPE declaration, fixed edge cases Fred Drake <fdrake@acm.org> parents: 19410 diff changeset	220 self.handle_comment(rawdata[i+4: j])
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	221 j = match.end()
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	222 return j
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	223
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	224 # Internal -- parse processing instr, return end or -1 if not terminated
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	225 def parse_pi(self, i):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	226 rawdata = self.rawdata
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	227 assert rawdata[i:i+2] == '<?', 'unexpected call to parse_pi()'
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	228 match = piclose.search(rawdata, i+2) # >
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	229 if not match:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	230 return -1
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	231 j = match.start()
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	232 self.handle_pi(rawdata[i+2: j])
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	233 j = match.end()
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	234 return j
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	235
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	236 # Internal -- handle starttag, return end or -1 if not terminated
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	237 def parse_starttag(self, i):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	238 self.__starttag_text = None
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	239 endpos = self.check_for_whole_start_tag(i)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	240 if endpos < 0:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	241 return endpos
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	242 rawdata = self.rawdata
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	243 self.__starttag_text = rawdata[i:endpos]
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	244
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	245 # Now parse the data between i+1 and j into a tag and attrs
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	246 attrs = []
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	247 match = tagfind.match(rawdata, i+1)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	248 assert match, 'unexpected call to parse_starttag()'
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	249 k = match.end()
21158 1acc4ba73b9d Convert to using string methods instead of the string module. Fred Drake <fdrake@acm.org> parents: 20067 diff changeset	250 self.lasttag = tag = rawdata[i+1:k].lower()
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	251
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	252 while k < endpos:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	253 m = attrfind.match(rawdata, k)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	254 if not m:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	255 break
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	256 attrname, rest, attrvalue = m.group(1, 2, 3)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	257 if not rest:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	258 attrvalue = None
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	259 elif attrvalue[:1] == '\'' == attrvalue[-1:] or \
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	260 attrvalue[:1] == '"' == attrvalue[-1:]:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	261 attrvalue = attrvalue[1:-1]
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	262 attrvalue = self.unescape(attrvalue)
21158 1acc4ba73b9d Convert to using string methods instead of the string module. Fred Drake <fdrake@acm.org> parents: 20067 diff changeset	263 attrs.append((attrname.lower(), attrvalue))
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	264 k = m.end()
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	265
21158 1acc4ba73b9d Convert to using string methods instead of the string module. Fred Drake <fdrake@acm.org> parents: 20067 diff changeset	266 end = rawdata[k:endpos].strip()
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	267 if end not in (">", "/>"):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	268 lineno, offset = self.getpos()
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	269 if "\n" in self.__starttag_text:
21158 1acc4ba73b9d Convert to using string methods instead of the string module. Fred Drake <fdrake@acm.org> parents: 20067 diff changeset	270 lineno = lineno + self.__starttag_text.count("\n")
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	271 offset = len(self.__starttag_text) \
21158 1acc4ba73b9d Convert to using string methods instead of the string module. Fred Drake <fdrake@acm.org> parents: 20067 diff changeset	272 - self.__starttag_text.rfind("\n")
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	273 else:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	274 offset = offset + len(self.__starttag_text)
20067 299f5fcb0ba8 Re-factor the HTMLParser class to use the new markupbase.ParserBase class. Fred Drake <fdrake@acm.org> parents: 19953 diff changeset	275 self.error("junk characters in start tag: %s"
299f5fcb0ba8 Re-factor the HTMLParser class to use the new markupbase.ParserBase class. Fred Drake <fdrake@acm.org> parents: 19953 diff changeset	276 % `rawdata[k:endpos][:20]`)
21158 1acc4ba73b9d Convert to using string methods instead of the string module. Fred Drake <fdrake@acm.org> parents: 20067 diff changeset	277 if end.endswith('/>'):
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	278 # XHTML-style empty tag: <span attr="value" />
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	279 self.handle_startendtag(tag, attrs)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	280 else:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	281 self.handle_starttag(tag, attrs)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	282 if tag in self.CDATA_CONTENT_ELEMENTS:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	283 self.set_cdata_mode()
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	284 return endpos
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	285
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	286 # Internal -- check to see if we have a complete starttag; return end
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	287 # or -1 if incomplete.
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	288 def check_for_whole_start_tag(self, i):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	289 rawdata = self.rawdata
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	290 m = locatestarttagend.match(rawdata, i)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	291 if m:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	292 j = m.end()
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	293 next = rawdata[j:j+1]
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	294 if next == ">":
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	295 return j + 1
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	296 if next == "/":
21158 1acc4ba73b9d Convert to using string methods instead of the string module. Fred Drake <fdrake@acm.org> parents: 20067 diff changeset	297 if rawdata.startswith("/>", j):
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	298 return j + 2
21158 1acc4ba73b9d Convert to using string methods instead of the string module. Fred Drake <fdrake@acm.org> parents: 20067 diff changeset	299 if rawdata.startswith("/", j):
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	300 # buffer boundary
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	301 return -1
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	302 # else bogus input
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	303 self.updatepos(i, j + 1)
20067 299f5fcb0ba8 Re-factor the HTMLParser class to use the new markupbase.ParserBase class. Fred Drake <fdrake@acm.org> parents: 19953 diff changeset	304 self.error("malformed empty start tag")
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	305 if next == "":
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	306 # end of input
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	307 return -1
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	308 if next in ("abcdefghijklmnopqrstuvwxyz=/"
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	309 "ABCDEFGHIJKLMNOPQRSTUVWXYZ"):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	310 # end of input in or before attribute value, or we have the
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	311 # '/' from a '/>' ending
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	312 return -1
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	313 self.updatepos(i, j)
20067 299f5fcb0ba8 Re-factor the HTMLParser class to use the new markupbase.ParserBase class. Fred Drake <fdrake@acm.org> parents: 19953 diff changeset	314 self.error("malformed start tag")
299f5fcb0ba8 Re-factor the HTMLParser class to use the new markupbase.ParserBase class. Fred Drake <fdrake@acm.org> parents: 19953 diff changeset	315 raise AssertionError("we should not get here!")
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	316
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	317 # Internal -- parse endtag, return end or -1 if incomplete
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	318 def parse_endtag(self, i):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	319 rawdata = self.rawdata
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	320 assert rawdata[i:i+2] == "</", "unexpected call to parse_endtag"
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	321 match = endendtag.search(rawdata, i+1) # >
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	322 if not match:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	323 return -1
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	324 j = match.end()
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	325 match = endtagfind.match(rawdata, i) # </ + tag + >
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	326 if not match:
20067 299f5fcb0ba8 Re-factor the HTMLParser class to use the new markupbase.ParserBase class. Fred Drake <fdrake@acm.org> parents: 19953 diff changeset	327 self.error("bad end tag: %s" % `rawdata[i:j]`)
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	328 tag = match.group(1)
21158 1acc4ba73b9d Convert to using string methods instead of the string module. Fred Drake <fdrake@acm.org> parents: 20067 diff changeset	329 self.handle_endtag(tag.lower())
23452 6bae0c31bbfb Simplify code to remove an unnecessary test. Fred Drake <fdrake@acm.org> parents: 21158 diff changeset	330 self.clear_cdata_mode()
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	331 return j
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	332
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	333 # Overridable -- finish processing of start+end tag: <tag.../>
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	334 def handle_startendtag(self, tag, attrs):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	335 self.handle_starttag(tag, attrs)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	336 self.handle_endtag(tag)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	337
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	338 # Overridable -- handle start tag
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	339 def handle_starttag(self, tag, attrs):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	340 pass
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	341
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	342 # Overridable -- handle end tag
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	343 def handle_endtag(self, tag):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	344 pass
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	345
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	346 # Overridable -- handle character reference
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	347 def handle_charref(self, name):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	348 pass
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	349
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	350 # Overridable -- handle entity reference
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	351 def handle_entityref(self, name):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	352 pass
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	353
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	354 # Overridable -- handle data
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	355 def handle_data(self, data):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	356 pass
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	357
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	358 # Overridable -- handle comment
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	359 def handle_comment(self, data):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	360 pass
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	361
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	362 # Overridable -- handle declaration
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	363 def handle_decl(self, decl):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	364 pass
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	365
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	366 # Overridable -- handle processing instruction
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	367 def handle_pi(self, data):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	368 pass
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	369
20067 299f5fcb0ba8 Re-factor the HTMLParser class to use the new markupbase.ParserBase class. Fred Drake <fdrake@acm.org> parents: 19953 diff changeset	370 def unknown_decl(self, data):
299f5fcb0ba8 Re-factor the HTMLParser class to use the new markupbase.ParserBase class. Fred Drake <fdrake@acm.org> parents: 19953 diff changeset	371 self.error("unknown declaration: " + `data`)
299f5fcb0ba8 Re-factor the HTMLParser class to use the new markupbase.ParserBase class. Fred Drake <fdrake@acm.org> parents: 19953 diff changeset	372
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	373 # Internal -- helper to remove special character quoting
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	374 def unescape(self, s):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	375 if '&' not in s:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	376 return s
21158 1acc4ba73b9d Convert to using string methods instead of the string module. Fred Drake <fdrake@acm.org> parents: 20067 diff changeset	377 s = s.replace("<", "<")
1acc4ba73b9d Convert to using string methods instead of the string module. Fred Drake <fdrake@acm.org> parents: 20067 diff changeset	378 s = s.replace(">", ">")
1acc4ba73b9d Convert to using string methods instead of the string module. Fred Drake <fdrake@acm.org> parents: 20067 diff changeset	379 s = s.replace("'", "'")
1acc4ba73b9d Convert to using string methods instead of the string module. Fred Drake <fdrake@acm.org> parents: 20067 diff changeset	380 s = s.replace(""", '"')
1acc4ba73b9d Convert to using string methods instead of the string module. Fred Drake <fdrake@acm.org> parents: 20067 diff changeset	381 s = s.replace("&", "&") # Must be last
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	382 return s

Mercurial > cpython-lru_cache-weakref

annotate Lib/HTMLParser.py @ 24042:63fbbed10714 legacy-trunk