cpython-lru_cache-weakref: Lib/HTMLParser.py annotate

annotate Lib/HTMLParser.py @ 25838:8540887bcec6 2.2

Backport of 2.183: Fix memory leak in add_subclass() found by NealN with valgrind.

author	Guido van Rossum <guido@python.org>
date	Fri, 18 Oct 2002 13:42:21 +0000
parents	1acc4ba73b9d
children	6bae0c31bbfb

rev	line source
18984 00935031b373 Change some comments into docstrings. Fred Drake <fdrake@acm.org> parents: 17934 diff changeset	1 """A parser for HTML and XHTML."""
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	2
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	3 # This file is based on sgmllib.py, but the API is slightly different.
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	4
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	5 # XXX There should be a way to distinguish between PCDATA (parsed
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	6 # character data -- the normal case), RCDATA (replaceable character
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	7 # data -- only char and entity references and end tags are special)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	8 # and CDATA (character data -- only end tags are special).
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	9
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	10
20067 299f5fcb0ba8 Re-factor the HTMLParser class to use the new markupbase.ParserBase class. Fred Drake <fdrake@acm.org> parents: 19953 diff changeset	11 import markupbase
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	12 import re
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	13
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	14 # Regular expressions used for parsing
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	15
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	16 interesting_normal = re.compile('[&<]')
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	17 interesting_cdata = re.compile(r'<(/\|\Z)')
19682 8dfb13a4e175 Added reasonable parsing of the DOCTYPE declaration, fixed edge cases Fred Drake <fdrake@acm.org> parents: 19410 diff changeset	18 incomplete = re.compile('&[a-zA-Z#]')
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	19
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	20 entityref = re.compile('&([a-zA-Z][-.a-zA-Z0-9]*)[^a-zA-Z0-9]')
18984 00935031b373 Change some comments into docstrings. Fred Drake <fdrake@acm.org> parents: 17934 diff changeset	21 charref = re.compile('&#(?:[0-9]+\|[xX][0-9a-fA-F]+)[^0-9a-fA-F]')
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	22
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	23 starttagopen = re.compile('<[a-zA-Z]')
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	24 piclose = re.compile('>')
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	25 commentclose = re.compile(r'--\s*>')
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	26 tagfind = re.compile('[a-zA-Z][-.a-zA-Z0-9:_]*')
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	27 attrfind = re.compile(
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	28 r'\s([a-zA-Z_][-.:a-zA-Z_0-9])(\s=\s'
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	29 r'(\'[^\']\'\|"[^"]"\|[-a-zA-Z0-9./:;+%?!&$_#=~]))?')
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	30
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	31 locatestarttagend = re.compile(r"""
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	32 <[a-zA-Z][-.a-zA-Z0-9:_]* # tag name
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	33 (?:\s+ # whitespace before attribute name
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	34 (?:[a-zA-Z_][-.:a-zA-Z0-9_]* # attribute name
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	35 (?:\s=\s # value indicator
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	36 (?:'[^']*' # LITA-enclosed value
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	37 \|\"[^\"]*\" # LIT-enclosed value
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	38 \|[^'\">\s]+ # bare value
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	39 )
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	40 )?
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	41 )
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	42 )*
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	43 \s* # trailing whitespace
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	44 """, re.VERBOSE)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	45 endendtag = re.compile('>')
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	46 endtagfind = re.compile('</\s([a-zA-Z][-.a-zA-Z0-9:_])\s*>')
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	47
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	48
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	49 class HTMLParseError(Exception):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	50 """Exception raised for all parse errors."""
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	51
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	52 def __init__(self, msg, position=(None, None)):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	53 assert msg
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	54 self.msg = msg
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	55 self.lineno = position[0]
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	56 self.offset = position[1]
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	57
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	58 def __str__(self):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	59 result = self.msg
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	60 if self.lineno is not None:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	61 result = result + ", at line %d" % self.lineno
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	62 if self.offset is not None:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	63 result = result + ", column %d" % (self.offset + 1)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	64 return result
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	65
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	66
20067 299f5fcb0ba8 Re-factor the HTMLParser class to use the new markupbase.ParserBase class. Fred Drake <fdrake@acm.org> parents: 19953 diff changeset	67 class HTMLParser(markupbase.ParserBase):
18984 00935031b373 Change some comments into docstrings. Fred Drake <fdrake@acm.org> parents: 17934 diff changeset	68 """Find tags and other markup and call handler functions.
00935031b373 Change some comments into docstrings. Fred Drake <fdrake@acm.org> parents: 17934 diff changeset	69
00935031b373 Change some comments into docstrings. Fred Drake <fdrake@acm.org> parents: 17934 diff changeset	70 Usage:
00935031b373 Change some comments into docstrings. Fred Drake <fdrake@acm.org> parents: 17934 diff changeset	71 p = HTMLParser()
00935031b373 Change some comments into docstrings. Fred Drake <fdrake@acm.org> parents: 17934 diff changeset	72 p.feed(data)
00935031b373 Change some comments into docstrings. Fred Drake <fdrake@acm.org> parents: 17934 diff changeset	73 ...
00935031b373 Change some comments into docstrings. Fred Drake <fdrake@acm.org> parents: 17934 diff changeset	74 p.close()
17932 ce7f0e380830 Removed incorrect comment left over from sgmllib.py. Guido van Rossum <guido@python.org> parents: 17853 diff changeset	75
18984 00935031b373 Change some comments into docstrings. Fred Drake <fdrake@acm.org> parents: 17934 diff changeset	76 Start tags are handled by calling self.handle_starttag() or
00935031b373 Change some comments into docstrings. Fred Drake <fdrake@acm.org> parents: 17934 diff changeset	77 self.handle_startendtag(); end tags by self.handle_endtag(). The
00935031b373 Change some comments into docstrings. Fred Drake <fdrake@acm.org> parents: 17934 diff changeset	78 data between tags is passed from the parser to the derived class
00935031b373 Change some comments into docstrings. Fred Drake <fdrake@acm.org> parents: 17934 diff changeset	79 by calling self.handle_data() with the data as argument (the data
00935031b373 Change some comments into docstrings. Fred Drake <fdrake@acm.org> parents: 17934 diff changeset	80 may be split up in arbitrary chunks). Entity references are
00935031b373 Change some comments into docstrings. Fred Drake <fdrake@acm.org> parents: 17934 diff changeset	81 passed by calling self.handle_entityref() with the entity
00935031b373 Change some comments into docstrings. Fred Drake <fdrake@acm.org> parents: 17934 diff changeset	82 reference as the argument. Numeric character references are
00935031b373 Change some comments into docstrings. Fred Drake <fdrake@acm.org> parents: 17934 diff changeset	83 passed to self.handle_charref() with the string containing the
00935031b373 Change some comments into docstrings. Fred Drake <fdrake@acm.org> parents: 17934 diff changeset	84 reference as the argument.
00935031b373 Change some comments into docstrings. Fred Drake <fdrake@acm.org> parents: 17934 diff changeset	85 """
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	86
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	87 CDATA_CONTENT_ELEMENTS = ("script", "style")
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	88
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	89
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	90 def __init__(self):
18984 00935031b373 Change some comments into docstrings. Fred Drake <fdrake@acm.org> parents: 17934 diff changeset	91 """Initialize and reset this instance."""
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	92 self.reset()
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	93
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	94 def reset(self):
18984 00935031b373 Change some comments into docstrings. Fred Drake <fdrake@acm.org> parents: 17934 diff changeset	95 """Reset this instance. Loses all unprocessed data."""
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	96 self.rawdata = ''
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	97 self.lasttag = '???'
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	98 self.interesting = interesting_normal
20067 299f5fcb0ba8 Re-factor the HTMLParser class to use the new markupbase.ParserBase class. Fred Drake <fdrake@acm.org> parents: 19953 diff changeset	99 markupbase.ParserBase.reset(self)
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	100
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	101 def feed(self, data):
18984 00935031b373 Change some comments into docstrings. Fred Drake <fdrake@acm.org> parents: 17934 diff changeset	102 """Feed data to the parser.
00935031b373 Change some comments into docstrings. Fred Drake <fdrake@acm.org> parents: 17934 diff changeset	103
00935031b373 Change some comments into docstrings. Fred Drake <fdrake@acm.org> parents: 17934 diff changeset	104 Call this as often as you want, with as little or as much text
00935031b373 Change some comments into docstrings. Fred Drake <fdrake@acm.org> parents: 17934 diff changeset	105 as you want (may include '\n').
00935031b373 Change some comments into docstrings. Fred Drake <fdrake@acm.org> parents: 17934 diff changeset	106 """
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	107 self.rawdata = self.rawdata + data
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	108 self.goahead(0)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	109
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	110 def close(self):
18984 00935031b373 Change some comments into docstrings. Fred Drake <fdrake@acm.org> parents: 17934 diff changeset	111 """Handle any buffered data."""
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	112 self.goahead(1)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	113
20067 299f5fcb0ba8 Re-factor the HTMLParser class to use the new markupbase.ParserBase class. Fred Drake <fdrake@acm.org> parents: 19953 diff changeset	114 def error(self, message):
299f5fcb0ba8 Re-factor the HTMLParser class to use the new markupbase.ParserBase class. Fred Drake <fdrake@acm.org> parents: 19953 diff changeset	115 raise HTMLParseError(message, self.getpos())
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	116
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	117 __starttag_text = None
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	118
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	119 def get_starttag_text(self):
18984 00935031b373 Change some comments into docstrings. Fred Drake <fdrake@acm.org> parents: 17934 diff changeset	120 """Return full source of start tag: '<...>'."""
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	121 return self.__starttag_text
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	122
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	123 def set_cdata_mode(self):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	124 self.interesting = interesting_cdata
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	125
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	126 def clear_cdata_mode(self):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	127 self.interesting = interesting_normal
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	128
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	129 # Internal -- handle data as far as reasonable. May leave state
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	130 # and data to be processed by a subsequent call. If 'end' is
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	131 # true, force handling all data as if followed by EOF marker.
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	132 def goahead(self, end):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	133 rawdata = self.rawdata
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	134 i = 0
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	135 n = len(rawdata)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	136 while i < n:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	137 match = self.interesting.search(rawdata, i) # < or &
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	138 if match:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	139 j = match.start()
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	140 else:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	141 j = n
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	142 if i < j: self.handle_data(rawdata[i:j])
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	143 i = self.updatepos(i, j)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	144 if i == n: break
21158 1acc4ba73b9d Convert to using string methods instead of the string module. Fred Drake <fdrake@acm.org> parents: 20067 diff changeset	145 startswith = rawdata.startswith
1acc4ba73b9d Convert to using string methods instead of the string module. Fred Drake <fdrake@acm.org> parents: 20067 diff changeset	146 if startswith('<', i):
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	147 if starttagopen.match(rawdata, i): # < + letter
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	148 k = self.parse_starttag(i)
21158 1acc4ba73b9d Convert to using string methods instead of the string module. Fred Drake <fdrake@acm.org> parents: 20067 diff changeset	149 elif startswith("</", i):
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	150 k = self.parse_endtag(i)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	151 if k >= 0:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	152 self.clear_cdata_mode()
21158 1acc4ba73b9d Convert to using string methods instead of the string module. Fred Drake <fdrake@acm.org> parents: 20067 diff changeset	153 elif startswith("<!--", i):
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	154 k = self.parse_comment(i)
21158 1acc4ba73b9d Convert to using string methods instead of the string module. Fred Drake <fdrake@acm.org> parents: 20067 diff changeset	155 elif startswith("<?", i):
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	156 k = self.parse_pi(i)
21158 1acc4ba73b9d Convert to using string methods instead of the string module. Fred Drake <fdrake@acm.org> parents: 20067 diff changeset	157 elif startswith("<!", i):
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	158 k = self.parse_declaration(i)
19682 8dfb13a4e175 Added reasonable parsing of the DOCTYPE declaration, fixed edge cases Fred Drake <fdrake@acm.org> parents: 19410 diff changeset	159 elif (i + 1) < n:
19410 c403af003884 Deal more appropriately with bare ampersands and pointy brackets; this Fred Drake <fdrake@acm.org> parents: 18984 diff changeset	160 self.handle_data("<")
c403af003884 Deal more appropriately with bare ampersands and pointy brackets; this Fred Drake <fdrake@acm.org> parents: 18984 diff changeset	161 k = i + 1
19682 8dfb13a4e175 Added reasonable parsing of the DOCTYPE declaration, fixed edge cases Fred Drake <fdrake@acm.org> parents: 19410 diff changeset	162 else:
8dfb13a4e175 Added reasonable parsing of the DOCTYPE declaration, fixed edge cases Fred Drake <fdrake@acm.org> parents: 19410 diff changeset	163 break
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	164 if k < 0:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	165 if end:
20067 299f5fcb0ba8 Re-factor the HTMLParser class to use the new markupbase.ParserBase class. Fred Drake <fdrake@acm.org> parents: 19953 diff changeset	166 self.error("EOF in middle of construct")
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	167 break
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	168 i = self.updatepos(i, k)
21158 1acc4ba73b9d Convert to using string methods instead of the string module. Fred Drake <fdrake@acm.org> parents: 20067 diff changeset	169 elif startswith("&#", i):
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	170 match = charref.match(rawdata, i)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	171 if match:
18984 00935031b373 Change some comments into docstrings. Fred Drake <fdrake@acm.org> parents: 17934 diff changeset	172 name = match.group()[2:-1]
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	173 self.handle_charref(name)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	174 k = match.end()
21158 1acc4ba73b9d Convert to using string methods instead of the string module. Fred Drake <fdrake@acm.org> parents: 20067 diff changeset	175 if not startswith(';', k-1):
19410 c403af003884 Deal more appropriately with bare ampersands and pointy brackets; this Fred Drake <fdrake@acm.org> parents: 18984 diff changeset	176 k = k - 1
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	177 i = self.updatepos(i, k)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	178 continue
19682 8dfb13a4e175 Added reasonable parsing of the DOCTYPE declaration, fixed edge cases Fred Drake <fdrake@acm.org> parents: 19410 diff changeset	179 else:
8dfb13a4e175 Added reasonable parsing of the DOCTYPE declaration, fixed edge cases Fred Drake <fdrake@acm.org> parents: 19410 diff changeset	180 break
21158 1acc4ba73b9d Convert to using string methods instead of the string module. Fred Drake <fdrake@acm.org> parents: 20067 diff changeset	181 elif startswith('&', i):
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	182 match = entityref.match(rawdata, i)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	183 if match:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	184 name = match.group(1)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	185 self.handle_entityref(name)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	186 k = match.end()
21158 1acc4ba73b9d Convert to using string methods instead of the string module. Fred Drake <fdrake@acm.org> parents: 20067 diff changeset	187 if not startswith(';', k-1):
19410 c403af003884 Deal more appropriately with bare ampersands and pointy brackets; this Fred Drake <fdrake@acm.org> parents: 18984 diff changeset	188 k = k - 1
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	189 i = self.updatepos(i, k)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	190 continue
19410 c403af003884 Deal more appropriately with bare ampersands and pointy brackets; this Fred Drake <fdrake@acm.org> parents: 18984 diff changeset	191 match = incomplete.match(rawdata, i)
c403af003884 Deal more appropriately with bare ampersands and pointy brackets; this Fred Drake <fdrake@acm.org> parents: 18984 diff changeset	192 if match:
19682 8dfb13a4e175 Added reasonable parsing of the DOCTYPE declaration, fixed edge cases Fred Drake <fdrake@acm.org> parents: 19410 diff changeset	193 # match.group() will contain at least 2 chars
21158 1acc4ba73b9d Convert to using string methods instead of the string module. Fred Drake <fdrake@acm.org> parents: 20067 diff changeset	194 if end and match.group() == rawdata[i:]:
20067 299f5fcb0ba8 Re-factor the HTMLParser class to use the new markupbase.ParserBase class. Fred Drake <fdrake@acm.org> parents: 19953 diff changeset	195 self.error("EOF in middle of entity or char ref")
19682 8dfb13a4e175 Added reasonable parsing of the DOCTYPE declaration, fixed edge cases Fred Drake <fdrake@acm.org> parents: 19410 diff changeset	196 # incomplete
8dfb13a4e175 Added reasonable parsing of the DOCTYPE declaration, fixed edge cases Fred Drake <fdrake@acm.org> parents: 19410 diff changeset	197 break
8dfb13a4e175 Added reasonable parsing of the DOCTYPE declaration, fixed edge cases Fred Drake <fdrake@acm.org> parents: 19410 diff changeset	198 elif (i + 1) < n:
8dfb13a4e175 Added reasonable parsing of the DOCTYPE declaration, fixed edge cases Fred Drake <fdrake@acm.org> parents: 19410 diff changeset	199 # not the end of the buffer, and can't be confused
8dfb13a4e175 Added reasonable parsing of the DOCTYPE declaration, fixed edge cases Fred Drake <fdrake@acm.org> parents: 19410 diff changeset	200 # with some other construct
8dfb13a4e175 Added reasonable parsing of the DOCTYPE declaration, fixed edge cases Fred Drake <fdrake@acm.org> parents: 19410 diff changeset	201 self.handle_data("&")
8dfb13a4e175 Added reasonable parsing of the DOCTYPE declaration, fixed edge cases Fred Drake <fdrake@acm.org> parents: 19410 diff changeset	202 i = self.updatepos(i, i + 1)
8dfb13a4e175 Added reasonable parsing of the DOCTYPE declaration, fixed edge cases Fred Drake <fdrake@acm.org> parents: 19410 diff changeset	203 else:
8dfb13a4e175 Added reasonable parsing of the DOCTYPE declaration, fixed edge cases Fred Drake <fdrake@acm.org> parents: 19410 diff changeset	204 break
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	205 else:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	206 assert 0, "interesting.search() lied"
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	207 # end while
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	208 if end and i < n:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	209 self.handle_data(rawdata[i:n])
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	210 i = self.updatepos(i, n)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	211 self.rawdata = rawdata[i:]
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	212
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	213 # Internal -- parse comment, return end or -1 if not terminated
19682 8dfb13a4e175 Added reasonable parsing of the DOCTYPE declaration, fixed edge cases Fred Drake <fdrake@acm.org> parents: 19410 diff changeset	214 def parse_comment(self, i, report=1):
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	215 rawdata = self.rawdata
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	216 assert rawdata[i:i+4] == '<!--', 'unexpected call to parse_comment()'
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	217 match = commentclose.search(rawdata, i+4)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	218 if not match:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	219 return -1
19682 8dfb13a4e175 Added reasonable parsing of the DOCTYPE declaration, fixed edge cases Fred Drake <fdrake@acm.org> parents: 19410 diff changeset	220 if report:
8dfb13a4e175 Added reasonable parsing of the DOCTYPE declaration, fixed edge cases Fred Drake <fdrake@acm.org> parents: 19410 diff changeset	221 j = match.start()
8dfb13a4e175 Added reasonable parsing of the DOCTYPE declaration, fixed edge cases Fred Drake <fdrake@acm.org> parents: 19410 diff changeset	222 self.handle_comment(rawdata[i+4: j])
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	223 j = match.end()
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	224 return j
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	225
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	226 # Internal -- parse processing instr, return end or -1 if not terminated
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	227 def parse_pi(self, i):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	228 rawdata = self.rawdata
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	229 assert rawdata[i:i+2] == '<?', 'unexpected call to parse_pi()'
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	230 match = piclose.search(rawdata, i+2) # >
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	231 if not match:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	232 return -1
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	233 j = match.start()
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	234 self.handle_pi(rawdata[i+2: j])
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	235 j = match.end()
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	236 return j
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	237
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	238 # Internal -- handle starttag, return end or -1 if not terminated
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	239 def parse_starttag(self, i):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	240 self.__starttag_text = None
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	241 endpos = self.check_for_whole_start_tag(i)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	242 if endpos < 0:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	243 return endpos
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	244 rawdata = self.rawdata
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	245 self.__starttag_text = rawdata[i:endpos]
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	246
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	247 # Now parse the data between i+1 and j into a tag and attrs
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	248 attrs = []
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	249 match = tagfind.match(rawdata, i+1)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	250 assert match, 'unexpected call to parse_starttag()'
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	251 k = match.end()
21158 1acc4ba73b9d Convert to using string methods instead of the string module. Fred Drake <fdrake@acm.org> parents: 20067 diff changeset	252 self.lasttag = tag = rawdata[i+1:k].lower()
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	253
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	254 while k < endpos:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	255 m = attrfind.match(rawdata, k)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	256 if not m:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	257 break
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	258 attrname, rest, attrvalue = m.group(1, 2, 3)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	259 if not rest:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	260 attrvalue = None
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	261 elif attrvalue[:1] == '\'' == attrvalue[-1:] or \
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	262 attrvalue[:1] == '"' == attrvalue[-1:]:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	263 attrvalue = attrvalue[1:-1]
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	264 attrvalue = self.unescape(attrvalue)
21158 1acc4ba73b9d Convert to using string methods instead of the string module. Fred Drake <fdrake@acm.org> parents: 20067 diff changeset	265 attrs.append((attrname.lower(), attrvalue))
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	266 k = m.end()
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	267
21158 1acc4ba73b9d Convert to using string methods instead of the string module. Fred Drake <fdrake@acm.org> parents: 20067 diff changeset	268 end = rawdata[k:endpos].strip()
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	269 if end not in (">", "/>"):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	270 lineno, offset = self.getpos()
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	271 if "\n" in self.__starttag_text:
21158 1acc4ba73b9d Convert to using string methods instead of the string module. Fred Drake <fdrake@acm.org> parents: 20067 diff changeset	272 lineno = lineno + self.__starttag_text.count("\n")
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	273 offset = len(self.__starttag_text) \
21158 1acc4ba73b9d Convert to using string methods instead of the string module. Fred Drake <fdrake@acm.org> parents: 20067 diff changeset	274 - self.__starttag_text.rfind("\n")
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	275 else:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	276 offset = offset + len(self.__starttag_text)
20067 299f5fcb0ba8 Re-factor the HTMLParser class to use the new markupbase.ParserBase class. Fred Drake <fdrake@acm.org> parents: 19953 diff changeset	277 self.error("junk characters in start tag: %s"
299f5fcb0ba8 Re-factor the HTMLParser class to use the new markupbase.ParserBase class. Fred Drake <fdrake@acm.org> parents: 19953 diff changeset	278 % `rawdata[k:endpos][:20]`)
21158 1acc4ba73b9d Convert to using string methods instead of the string module. Fred Drake <fdrake@acm.org> parents: 20067 diff changeset	279 if end.endswith('/>'):
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	280 # XHTML-style empty tag: <span attr="value" />
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	281 self.handle_startendtag(tag, attrs)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	282 else:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	283 self.handle_starttag(tag, attrs)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	284 if tag in self.CDATA_CONTENT_ELEMENTS:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	285 self.set_cdata_mode()
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	286 return endpos
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	287
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	288 # Internal -- check to see if we have a complete starttag; return end
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	289 # or -1 if incomplete.
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	290 def check_for_whole_start_tag(self, i):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	291 rawdata = self.rawdata
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	292 m = locatestarttagend.match(rawdata, i)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	293 if m:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	294 j = m.end()
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	295 next = rawdata[j:j+1]
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	296 if next == ">":
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	297 return j + 1
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	298 if next == "/":
21158 1acc4ba73b9d Convert to using string methods instead of the string module. Fred Drake <fdrake@acm.org> parents: 20067 diff changeset	299 if rawdata.startswith("/>", j):
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	300 return j + 2
21158 1acc4ba73b9d Convert to using string methods instead of the string module. Fred Drake <fdrake@acm.org> parents: 20067 diff changeset	301 if rawdata.startswith("/", j):
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	302 # buffer boundary
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	303 return -1
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	304 # else bogus input
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	305 self.updatepos(i, j + 1)
20067 299f5fcb0ba8 Re-factor the HTMLParser class to use the new markupbase.ParserBase class. Fred Drake <fdrake@acm.org> parents: 19953 diff changeset	306 self.error("malformed empty start tag")
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	307 if next == "":
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	308 # end of input
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	309 return -1
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	310 if next in ("abcdefghijklmnopqrstuvwxyz=/"
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	311 "ABCDEFGHIJKLMNOPQRSTUVWXYZ"):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	312 # end of input in or before attribute value, or we have the
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	313 # '/' from a '/>' ending
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	314 return -1
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	315 self.updatepos(i, j)
20067 299f5fcb0ba8 Re-factor the HTMLParser class to use the new markupbase.ParserBase class. Fred Drake <fdrake@acm.org> parents: 19953 diff changeset	316 self.error("malformed start tag")
299f5fcb0ba8 Re-factor the HTMLParser class to use the new markupbase.ParserBase class. Fred Drake <fdrake@acm.org> parents: 19953 diff changeset	317 raise AssertionError("we should not get here!")
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	318
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	319 # Internal -- parse endtag, return end or -1 if incomplete
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	320 def parse_endtag(self, i):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	321 rawdata = self.rawdata
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	322 assert rawdata[i:i+2] == "</", "unexpected call to parse_endtag"
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	323 match = endendtag.search(rawdata, i+1) # >
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	324 if not match:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	325 return -1
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	326 j = match.end()
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	327 match = endtagfind.match(rawdata, i) # </ + tag + >
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	328 if not match:
20067 299f5fcb0ba8 Re-factor the HTMLParser class to use the new markupbase.ParserBase class. Fred Drake <fdrake@acm.org> parents: 19953 diff changeset	329 self.error("bad end tag: %s" % `rawdata[i:j]`)
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	330 tag = match.group(1)
21158 1acc4ba73b9d Convert to using string methods instead of the string module. Fred Drake <fdrake@acm.org> parents: 20067 diff changeset	331 self.handle_endtag(tag.lower())
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	332 return j
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	333
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	334 # Overridable -- finish processing of start+end tag: <tag.../>
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	335 def handle_startendtag(self, tag, attrs):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	336 self.handle_starttag(tag, attrs)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	337 self.handle_endtag(tag)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	338
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	339 # Overridable -- handle start tag
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	340 def handle_starttag(self, tag, attrs):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	341 pass
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	342
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	343 # Overridable -- handle end tag
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	344 def handle_endtag(self, tag):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	345 pass
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	346
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	347 # Overridable -- handle character reference
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	348 def handle_charref(self, name):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	349 pass
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	350
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	351 # Overridable -- handle entity reference
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	352 def handle_entityref(self, name):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	353 pass
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	354
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	355 # Overridable -- handle data
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	356 def handle_data(self, data):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	357 pass
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	358
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	359 # Overridable -- handle comment
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	360 def handle_comment(self, data):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	361 pass
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	362
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	363 # Overridable -- handle declaration
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	364 def handle_decl(self, decl):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	365 pass
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	366
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	367 # Overridable -- handle processing instruction
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	368 def handle_pi(self, data):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	369 pass
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	370
20067 299f5fcb0ba8 Re-factor the HTMLParser class to use the new markupbase.ParserBase class. Fred Drake <fdrake@acm.org> parents: 19953 diff changeset	371 def unknown_decl(self, data):
299f5fcb0ba8 Re-factor the HTMLParser class to use the new markupbase.ParserBase class. Fred Drake <fdrake@acm.org> parents: 19953 diff changeset	372 self.error("unknown declaration: " + `data`)
299f5fcb0ba8 Re-factor the HTMLParser class to use the new markupbase.ParserBase class. Fred Drake <fdrake@acm.org> parents: 19953 diff changeset	373
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	374 # Internal -- helper to remove special character quoting
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	375 def unescape(self, s):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	376 if '&' not in s:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	377 return s
21158 1acc4ba73b9d Convert to using string methods instead of the string module. Fred Drake <fdrake@acm.org> parents: 20067 diff changeset	378 s = s.replace("<", "<")
1acc4ba73b9d Convert to using string methods instead of the string module. Fred Drake <fdrake@acm.org> parents: 20067 diff changeset	379 s = s.replace(">", ">")
1acc4ba73b9d Convert to using string methods instead of the string module. Fred Drake <fdrake@acm.org> parents: 20067 diff changeset	380 s = s.replace("'", "'")
1acc4ba73b9d Convert to using string methods instead of the string module. Fred Drake <fdrake@acm.org> parents: 20067 diff changeset	381 s = s.replace(""", '"')
1acc4ba73b9d Convert to using string methods instead of the string module. Fred Drake <fdrake@acm.org> parents: 20067 diff changeset	382 s = s.replace("&", "&") # Must be last
17853 055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is Guido van Rossum <guido@python.org> parents: diff changeset	383 return s

Mercurial > cpython-lru_cache-weakref

annotate Lib/HTMLParser.py @ 25838:8540887bcec6 2.2