Mercurial > cpython-lru_cache-weakref

annotate Lib/HTMLParser.py @ 17853:055fc046e080 legacy-trunk

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
A much improved HTML parser -- a replacement for sgmllib. The API is derived from but not quite compatible with that of sgmllib, so it's a new file. I suppose it needs documentation, and htmllib needs to be changed to use this instead of sgmllib, and sgmllib needs to be declared obsolete. But that can all be done later. This code was first published as part of TAL (part of Zope Page Templates), but that was strongly based on sgmllib anyway. Authors are Fred drake and Guido van Rossum.
author Guido van Rossum <guido@python.org>
date Fri, 18 May 2001 14:50:52 +0000
parents
children ce7f0e380830
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
17853
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
1 """A parser for HTML."""
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
2
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
3 # This file is based on sgmllib.py, but the API is slightly different.
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
4
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
5 # XXX There should be a way to distinguish between PCDATA (parsed
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
6 # character data -- the normal case), RCDATA (replaceable character
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
7 # data -- only char and entity references and end tags are special)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
8 # and CDATA (character data -- only end tags are special).
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
9
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
10
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
11 import re
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
12 import string
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
13
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
14 # Regular expressions used for parsing
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
15
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
16 interesting_normal = re.compile('[&<]')
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
17 interesting_cdata = re.compile(r'<(/|\Z)')
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
18 incomplete = re.compile('&([a-zA-Z][-.a-zA-Z0-9]*|#[0-9]*)?')
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
19
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
20 entityref = re.compile('&([a-zA-Z][-.a-zA-Z0-9]*)[^a-zA-Z0-9]')
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
21 charref = re.compile('&#([0-9]+)[^0-9]')
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
22
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
23 starttagopen = re.compile('<[a-zA-Z]')
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
24 piopen = re.compile(r'<\?')
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
25 piclose = re.compile('>')
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
26 endtagopen = re.compile('</')
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
27 declopen = re.compile('<!')
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
28 special = re.compile('<![^<>]*>')
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
29 commentopen = re.compile('<!--')
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
30 commentclose = re.compile(r'--\s*>')
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
31 tagfind = re.compile('[a-zA-Z][-.a-zA-Z0-9:_]*')
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
32 attrfind = re.compile(
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
33 r'\s*([a-zA-Z_][-.:a-zA-Z_0-9]*)(\s*=\s*'
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
34 r'(\'[^\']*\'|"[^"]*"|[-a-zA-Z0-9./:;+*%?!&$\(\)_#=~]*))?')
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
35
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
36 locatestarttagend = re.compile(r"""
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
37 <[a-zA-Z][-.a-zA-Z0-9:_]* # tag name
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
38 (?:\s+ # whitespace before attribute name
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
39 (?:[a-zA-Z_][-.:a-zA-Z0-9_]* # attribute name
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
40 (?:\s*=\s* # value indicator
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
41 (?:'[^']*' # LITA-enclosed value
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
42 |\"[^\"]*\" # LIT-enclosed value
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
43 |[^'\">\s]+ # bare value
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
44 )
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
45 )?
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
46 )
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
47 )*
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
48 \s* # trailing whitespace
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
49 """, re.VERBOSE)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
50 endstarttag = re.compile(r"\s*/?>")
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
51 endendtag = re.compile('>')
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
52 endtagfind = re.compile('</\s*([a-zA-Z][-.a-zA-Z0-9:_]*)\s*>')
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
53
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
54 declname = re.compile(r'[a-zA-Z][-_.a-zA-Z0-9]*\s*')
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
55 declstringlit = re.compile(r'(\'[^\']*\'|"[^"]*")\s*')
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
56
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
57
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
58 class HTMLParseError(Exception):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
59 """Exception raised for all parse errors."""
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
60
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
61 def __init__(self, msg, position=(None, None)):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
62 assert msg
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
63 self.msg = msg
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
64 self.lineno = position[0]
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
65 self.offset = position[1]
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
66
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
67 def __str__(self):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
68 result = self.msg
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
69 if self.lineno is not None:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
70 result = result + ", at line %d" % self.lineno
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
71 if self.offset is not None:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
72 result = result + ", column %d" % (self.offset + 1)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
73 return result
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
74
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
75
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
76 # HTML parser class -- find tags and call handler functions.
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
77 # Usage: p = HTMLParser(); p.feed(data); ...; p.close().
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
78 # The dtd is defined by deriving a class which defines methods
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
79 # with special names to handle tags: start_foo and end_foo to handle
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
80 # <foo> and </foo>, respectively, or do_foo to handle <foo> by itself.
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
81 # (Tags are converted to lower case for this purpose.) The data
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
82 # between tags is passed to the parser by calling self.handle_data()
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
83 # with some data as argument (the data may be split up in arbitrary
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
84 # chunks). Entity references are passed by calling
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
85 # self.handle_entityref() with the entity reference as argument.
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
86
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
87 class HTMLParser:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
88
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
89 CDATA_CONTENT_ELEMENTS = ("script", "style")
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
90
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
91
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
92 # Interface -- initialize and reset this instance
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
93 def __init__(self):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
94 self.reset()
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
95
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
96 # Interface -- reset this instance. Loses all unprocessed data
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
97 def reset(self):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
98 self.rawdata = ''
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
99 self.stack = []
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
100 self.lasttag = '???'
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
101 self.lineno = 1
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
102 self.offset = 0
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
103 self.interesting = interesting_normal
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
104
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
105 # Interface -- feed some data to the parser. Call this as
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
106 # often as you want, with as little or as much text as you
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
107 # want (may include '\n'). (This just saves the text, all the
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
108 # processing is done by goahead().)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
109 def feed(self, data):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
110 self.rawdata = self.rawdata + data
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
111 self.goahead(0)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
112
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
113 # Interface -- handle the remaining data
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
114 def close(self):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
115 self.goahead(1)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
116
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
117 # Internal -- update line number and offset. This should be
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
118 # called for each piece of data exactly once, in order -- in other
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
119 # words the concatenation of all the input strings to this
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
120 # function should be exactly the entire input.
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
121 def updatepos(self, i, j):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
122 if i >= j:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
123 return j
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
124 rawdata = self.rawdata
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
125 nlines = string.count(rawdata, "\n", i, j)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
126 if nlines:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
127 self.lineno = self.lineno + nlines
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
128 pos = string.rindex(rawdata, "\n", i, j) # Should not fail
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
129 self.offset = j-(pos+1)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
130 else:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
131 self.offset = self.offset + j-i
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
132 return j
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
133
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
134 # Interface -- return current line number and offset.
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
135 def getpos(self):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
136 return self.lineno, self.offset
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
137
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
138 __starttag_text = None
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
139
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
140 # Interface -- return full source of start tag: "<...>"
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
141 def get_starttag_text(self):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
142 return self.__starttag_text
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
143
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
144 def set_cdata_mode(self):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
145 self.interesting = interesting_cdata
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
146
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
147 def clear_cdata_mode(self):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
148 self.interesting = interesting_normal
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
149
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
150 # Internal -- handle data as far as reasonable. May leave state
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
151 # and data to be processed by a subsequent call. If 'end' is
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
152 # true, force handling all data as if followed by EOF marker.
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
153 def goahead(self, end):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
154 rawdata = self.rawdata
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
155 i = 0
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
156 n = len(rawdata)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
157 while i < n:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
158 match = self.interesting.search(rawdata, i) # < or &
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
159 if match:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
160 j = match.start()
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
161 else:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
162 j = n
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
163 if i < j: self.handle_data(rawdata[i:j])
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
164 i = self.updatepos(i, j)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
165 if i == n: break
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
166 if rawdata[i] == '<':
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
167 if starttagopen.match(rawdata, i): # < + letter
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
168 k = self.parse_starttag(i)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
169 elif endtagopen.match(rawdata, i): # </
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
170 k = self.parse_endtag(i)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
171 if k >= 0:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
172 self.clear_cdata_mode()
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
173 elif commentopen.match(rawdata, i): # <!--
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
174 k = self.parse_comment(i)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
175 elif piopen.match(rawdata, i): # <?
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
176 k = self.parse_pi(i)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
177 elif declopen.match(rawdata, i): # <!
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
178 k = self.parse_declaration(i)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
179 else:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
180 if i < n-1:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
181 raise HTMLParseError(
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
182 "invalid '<' construct: %s" % `rawdata[i:i+2]`,
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
183 self.getpos())
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
184 k = -1
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
185 if k < 0:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
186 if end:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
187 raise HTMLParseError("EOF in middle of construct",
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
188 self.getpos())
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
189 break
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
190 i = self.updatepos(i, k)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
191 elif rawdata[i] == '&':
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
192 match = charref.match(rawdata, i)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
193 if match:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
194 name = match.group(1)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
195 self.handle_charref(name)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
196 k = match.end()
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
197 if rawdata[k-1] != ';':
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
198 k = k-1
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
199 i = self.updatepos(i, k)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
200 continue
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
201 match = entityref.match(rawdata, i)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
202 if match:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
203 name = match.group(1)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
204 self.handle_entityref(name)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
205 k = match.end()
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
206 if rawdata[k-1] != ';':
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
207 k = k-1
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
208 i = self.updatepos(i, k)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
209 continue
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
210 if incomplete.match(rawdata, i):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
211 if end:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
212 raise HTMLParseError(
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
213 "EOF in middle of entity or char ref",
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
214 self.getpos())
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
215 return -1 # incomplete
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
216 raise HTMLParseError("'&' not part of entity or char ref",
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
217 self.getpos())
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
218 else:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
219 assert 0, "interesting.search() lied"
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
220 # end while
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
221 if end and i < n:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
222 self.handle_data(rawdata[i:n])
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
223 i = self.updatepos(i, n)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
224 self.rawdata = rawdata[i:]
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
225
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
226 # Internal -- parse comment, return end or -1 if not terminated
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
227 def parse_comment(self, i):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
228 rawdata = self.rawdata
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
229 assert rawdata[i:i+4] == '<!--', 'unexpected call to parse_comment()'
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
230 match = commentclose.search(rawdata, i+4)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
231 if not match:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
232 return -1
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
233 j = match.start()
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
234 self.handle_comment(rawdata[i+4: j])
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
235 j = match.end()
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
236 return j
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
237
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
238 # Internal -- parse declaration.
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
239 def parse_declaration(self, i):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
240 # This is some sort of declaration; in "HTML as
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
241 # deployed," this should only be the document type
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
242 # declaration ("<!DOCTYPE html...>").
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
243 rawdata = self.rawdata
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
244 j = i + 2
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
245 assert rawdata[i:j] == "<!", "unexpected call to parse_declaration"
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
246 if rawdata[j:j+1] in ("-", ""):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
247 # Start of comment followed by buffer boundary,
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
248 # or just a buffer boundary.
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
249 return -1
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
250 # in practice, this should look like: ((name|stringlit) S*)+ '>'
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
251 n = len(rawdata)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
252 while j < n:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
253 c = rawdata[j]
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
254 if c == ">":
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
255 # end of declaration syntax
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
256 self.handle_decl(rawdata[i+2:j])
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
257 return j + 1
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
258 if c in "\"'":
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
259 m = declstringlit.match(rawdata, j)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
260 if not m:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
261 return -1 # incomplete
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
262 j = m.end()
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
263 elif c in "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ":
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
264 m = declname.match(rawdata, j)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
265 if not m:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
266 return -1 # incomplete
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
267 j = m.end()
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
268 else:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
269 raise HTMLParseError(
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
270 "unexpected char in declaration: %s" % `rawdata[j]`,
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
271 self.getpos())
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
272 return -1 # incomplete
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
273
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
274 # Internal -- parse processing instr, return end or -1 if not terminated
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
275 def parse_pi(self, i):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
276 rawdata = self.rawdata
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
277 assert rawdata[i:i+2] == '<?', 'unexpected call to parse_pi()'
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
278 match = piclose.search(rawdata, i+2) # >
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
279 if not match:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
280 return -1
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
281 j = match.start()
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
282 self.handle_pi(rawdata[i+2: j])
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
283 j = match.end()
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
284 return j
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
285
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
286 # Internal -- handle starttag, return end or -1 if not terminated
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
287 def parse_starttag(self, i):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
288 self.__starttag_text = None
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
289 endpos = self.check_for_whole_start_tag(i)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
290 if endpos < 0:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
291 return endpos
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
292 rawdata = self.rawdata
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
293 self.__starttag_text = rawdata[i:endpos]
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
294
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
295 # Now parse the data between i+1 and j into a tag and attrs
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
296 attrs = []
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
297 match = tagfind.match(rawdata, i+1)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
298 assert match, 'unexpected call to parse_starttag()'
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
299 k = match.end()
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
300 self.lasttag = tag = string.lower(rawdata[i+1:k])
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
301
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
302 while k < endpos:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
303 m = attrfind.match(rawdata, k)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
304 if not m:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
305 break
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
306 attrname, rest, attrvalue = m.group(1, 2, 3)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
307 if not rest:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
308 attrvalue = None
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
309 elif attrvalue[:1] == '\'' == attrvalue[-1:] or \
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
310 attrvalue[:1] == '"' == attrvalue[-1:]:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
311 attrvalue = attrvalue[1:-1]
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
312 attrvalue = self.unescape(attrvalue)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
313 attrs.append((string.lower(attrname), attrvalue))
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
314 k = m.end()
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
315
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
316 end = string.strip(rawdata[k:endpos])
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
317 if end not in (">", "/>"):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
318 lineno, offset = self.getpos()
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
319 if "\n" in self.__starttag_text:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
320 lineno = lineno + string.count(self.__starttag_text, "\n")
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
321 offset = len(self.__starttag_text) \
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
322 - string.rfind(self.__starttag_text, "\n")
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
323 else:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
324 offset = offset + len(self.__starttag_text)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
325 raise HTMLParseError("junk characters in start tag: %s"
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
326 % `rawdata[k:endpos][:20]`,
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
327 (lineno, offset))
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
328 if end[-2:] == '/>':
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
329 # XHTML-style empty tag: <span attr="value" />
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
330 self.handle_startendtag(tag, attrs)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
331 else:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
332 self.handle_starttag(tag, attrs)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
333 if tag in self.CDATA_CONTENT_ELEMENTS:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
334 self.set_cdata_mode()
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
335 return endpos
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
336
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
337 # Internal -- check to see if we have a complete starttag; return end
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
338 # or -1 if incomplete.
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
339 def check_for_whole_start_tag(self, i):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
340 rawdata = self.rawdata
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
341 m = locatestarttagend.match(rawdata, i)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
342 if m:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
343 j = m.end()
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
344 next = rawdata[j:j+1]
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
345 if next == ">":
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
346 return j + 1
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
347 if next == "/":
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
348 s = rawdata[j:j+2]
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
349 if s == "/>":
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
350 return j + 2
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
351 if s == "/":
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
352 # buffer boundary
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
353 return -1
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
354 # else bogus input
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
355 self.updatepos(i, j + 1)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
356 raise HTMLParseError("malformed empty start tag",
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
357 self.getpos())
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
358 if next == "":
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
359 # end of input
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
360 return -1
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
361 if next in ("abcdefghijklmnopqrstuvwxyz=/"
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
362 "ABCDEFGHIJKLMNOPQRSTUVWXYZ"):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
363 # end of input in or before attribute value, or we have the
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
364 # '/' from a '/>' ending
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
365 return -1
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
366 self.updatepos(i, j)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
367 raise HTMLParseError("malformed start tag", self.getpos())
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
368 raise AssertionError("we should not gt here!")
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
369
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
370 # Internal -- parse endtag, return end or -1 if incomplete
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
371 def parse_endtag(self, i):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
372 rawdata = self.rawdata
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
373 assert rawdata[i:i+2] == "</", "unexpected call to parse_endtag"
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
374 match = endendtag.search(rawdata, i+1) # >
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
375 if not match:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
376 return -1
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
377 j = match.end()
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
378 match = endtagfind.match(rawdata, i) # </ + tag + >
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
379 if not match:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
380 raise HTMLParseError("bad end tag: %s" % `rawdata[i:j]`,
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
381 self.getpos())
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
382 tag = match.group(1)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
383 self.handle_endtag(string.lower(tag))
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
384 return j
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
385
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
386 # Overridable -- finish processing of start+end tag: <tag.../>
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
387 def handle_startendtag(self, tag, attrs):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
388 self.handle_starttag(tag, attrs)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
389 self.handle_endtag(tag)
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
390
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
391 # Overridable -- handle start tag
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
392 def handle_starttag(self, tag, attrs):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
393 pass
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
394
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
395 # Overridable -- handle end tag
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
396 def handle_endtag(self, tag):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
397 pass
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
398
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
399 # Overridable -- handle character reference
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
400 def handle_charref(self, name):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
401 pass
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
402
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
403 # Overridable -- handle entity reference
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
404 def handle_entityref(self, name):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
405 pass
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
406
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
407 # Overridable -- handle data
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
408 def handle_data(self, data):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
409 pass
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
410
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
411 # Overridable -- handle comment
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
412 def handle_comment(self, data):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
413 pass
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
414
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
415 # Overridable -- handle declaration
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
416 def handle_decl(self, decl):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
417 pass
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
418
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
419 # Overridable -- handle processing instruction
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
420 def handle_pi(self, data):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
421 pass
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
422
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
423 # Internal -- helper to remove special character quoting
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
424 def unescape(self, s):
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
425 if '&' not in s:
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
426 return s
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
427 s = string.replace(s, "&lt;", "<")
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
428 s = string.replace(s, "&gt;", ">")
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
429 s = string.replace(s, "&apos;", "'")
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
430 s = string.replace(s, "&quot;", '"')
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
431 s = string.replace(s, "&amp;", "&") # Must be last
055fc046e080 A much improved HTML parser -- a replacement for sgmllib. The API is
Guido van Rossum <guido@python.org>
parents:
diff changeset
432 return s