Mercurial > cpython-lru_cache-weakref

annotate Lib/_markupbase.py @ 84007:dbfbdf2b5c19 3.3

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Fix #17967: For ftp urls CWD to target instead of hopping to each directory towards target. This fixes a bug where target is accessible, but parent directories are restricted.
author Senthil Kumaran <senthil@uthcode.com>
date Sat, 01 Jun 2013 08:27:06 -0700
parents 4ddbb756b602
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
35386
38824b6e3600 add note about "markupbase" not being intended for direct use
Fred Drake <fdrake@acm.org>
parents: 32603
diff changeset
1 """Shared support for scanning document type declarations in HTML and XHTML.
38824b6e3600 add note about "markupbase" not being intended for direct use
Fred Drake <fdrake@acm.org>
parents: 32603
diff changeset
2
47434
f150177ff1ff Remove the htmllib and sgmllib modules as per PEP 3108.
Georg Brandl <georg@python.org>
parents: 44212
diff changeset
3 This module is used as a foundation for the html.parser module. It has no
f150177ff1ff Remove the htmllib and sgmllib modules as per PEP 3108.
Georg Brandl <georg@python.org>
parents: 44212
diff changeset
4 documented public API and should not be used directly.
35386
38824b6e3600 add note about "markupbase" not being intended for direct use
Fred Drake <fdrake@acm.org>
parents: 32603
diff changeset
5
38824b6e3600 add note about "markupbase" not being intended for direct use
Fred Drake <fdrake@acm.org>
parents: 32603
diff changeset
6 """
20065
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
7
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
8 import re
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
9
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
10 _declname_match = re.compile(r'[a-zA-Z][-_.a-zA-Z0-9]*\s*').match
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
11 _declstringlit_match = re.compile(r'(\'[^\']*\'|"[^"]*")\s*').match
28446
500f545a24f0 Patch #545300: Support marked sections.
Martin v. Löwis <martin@v.loewis.de>
parents: 23645
diff changeset
12 _commentclose = re.compile(r'--\s*>')
500f545a24f0 Patch #545300: Support marked sections.
Martin v. Löwis <martin@v.loewis.de>
parents: 23645
diff changeset
13 _markedsectionclose = re.compile(r']\s*]\s*>')
500f545a24f0 Patch #545300: Support marked sections.
Martin v. Löwis <martin@v.loewis.de>
parents: 23645
diff changeset
14
500f545a24f0 Patch #545300: Support marked sections.
Martin v. Löwis <martin@v.loewis.de>
parents: 23645
diff changeset
15 # An analysis of the MS-Word extensions is available at
500f545a24f0 Patch #545300: Support marked sections.
Martin v. Löwis <martin@v.loewis.de>
parents: 23645
diff changeset
16 # http://www.planetpublish.com/xmlarena/xap/Thursday/WordtoXML.pdf
500f545a24f0 Patch #545300: Support marked sections.
Martin v. Löwis <martin@v.loewis.de>
parents: 23645
diff changeset
17
500f545a24f0 Patch #545300: Support marked sections.
Martin v. Löwis <martin@v.loewis.de>
parents: 23645
diff changeset
18 _msmarkedsectionclose = re.compile(r']\s*>')
20065
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
19
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
20 del re
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
21
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
22
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
23 class ParserBase:
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
24 """Parser base class which provides some common support methods used
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
25 by the SGML/HTML and XHTML parsers."""
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
26
20699
fa0bf6c79e04 Re-arrange things and remove some unused variables/imports to keep pychecker
Fred Drake <fdrake@acm.org>
parents: 20416
diff changeset
27 def __init__(self):
fa0bf6c79e04 Re-arrange things and remove some unused variables/imports to keep pychecker
Fred Drake <fdrake@acm.org>
parents: 20416
diff changeset
28 if self.__class__ is ParserBase:
fa0bf6c79e04 Re-arrange things and remove some unused variables/imports to keep pychecker
Fred Drake <fdrake@acm.org>
parents: 20416
diff changeset
29 raise RuntimeError(
44212
c6fb393b6c81 rename markupbase to _markupbase
Fred Drake <fdrake@acm.org>
parents: 38673
diff changeset
30 "_markupbase.ParserBase must be subclassed")
20699
fa0bf6c79e04 Re-arrange things and remove some unused variables/imports to keep pychecker
Fred Drake <fdrake@acm.org>
parents: 20416
diff changeset
31
fa0bf6c79e04 Re-arrange things and remove some unused variables/imports to keep pychecker
Fred Drake <fdrake@acm.org>
parents: 20416
diff changeset
32 def error(self, message):
fa0bf6c79e04 Re-arrange things and remove some unused variables/imports to keep pychecker
Fred Drake <fdrake@acm.org>
parents: 20416
diff changeset
33 raise NotImplementedError(
fa0bf6c79e04 Re-arrange things and remove some unused variables/imports to keep pychecker
Fred Drake <fdrake@acm.org>
parents: 20416
diff changeset
34 "subclasses of ParserBase must override error()")
fa0bf6c79e04 Re-arrange things and remove some unused variables/imports to keep pychecker
Fred Drake <fdrake@acm.org>
parents: 20416
diff changeset
35
20065
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
36 def reset(self):
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
37 self.lineno = 1
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
38 self.offset = 0
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
39
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
40 def getpos(self):
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
41 """Return current line number and offset."""
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
42 return self.lineno, self.offset
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
43
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
44 # Internal -- update line number and offset. This should be
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
45 # called for each piece of data exactly once, in order -- in other
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
46 # words the concatenation of all the input strings to this
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
47 # function should be exactly the entire input.
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
48 def updatepos(self, i, j):
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
49 if i >= j:
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
50 return j
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
51 rawdata = self.rawdata
23590
a4bb18212b48 Use string methods where possible, and remove import string
Neal Norwitz <nnorwitz@gmail.com>
parents: 20699
diff changeset
52 nlines = rawdata.count("\n", i, j)
20065
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
53 if nlines:
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
54 self.lineno = self.lineno + nlines
23590
a4bb18212b48 Use string methods where possible, and remove import string
Neal Norwitz <nnorwitz@gmail.com>
parents: 20699
diff changeset
55 pos = rawdata.rindex("\n", i, j) # Should not fail
20065
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
56 self.offset = j-(pos+1)
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
57 else:
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
58 self.offset = self.offset + j-i
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
59 return j
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
60
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
61 _decl_otherchars = ''
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
62
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
63 # Internal -- parse declaration (for use by subclasses).
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
64 def parse_declaration(self, i):
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
65 # This is some sort of declaration; in "HTML as
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
66 # deployed," this should only be the document type
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
67 # declaration ("<!DOCTYPE html...>").
28774
c9bee11eaa8b Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 28446
diff changeset
68 # ISO 8879:1986, however, has more complex
28446
500f545a24f0 Patch #545300: Support marked sections.
Martin v. Löwis <martin@v.loewis.de>
parents: 23645
diff changeset
69 # declaration syntax for elements in <!...>, including:
500f545a24f0 Patch #545300: Support marked sections.
Martin v. Löwis <martin@v.loewis.de>
parents: 23645
diff changeset
70 # --comment--
500f545a24f0 Patch #545300: Support marked sections.
Martin v. Löwis <martin@v.loewis.de>
parents: 23645
diff changeset
71 # [marked section]
28774
c9bee11eaa8b Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 28446
diff changeset
72 # name in the following list: ENTITY, DOCTYPE, ELEMENT,
c9bee11eaa8b Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 28446
diff changeset
73 # ATTLIST, NOTATION, SHORTREF, USEMAP,
28446
500f545a24f0 Patch #545300: Support marked sections.
Martin v. Löwis <martin@v.loewis.de>
parents: 23645
diff changeset
74 # LINKTYPE, LINK, IDLINK, USELINK, SYSTEM
20065
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
75 rawdata = self.rawdata
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
76 j = i + 2
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
77 assert rawdata[i:j] == "<!", "unexpected call to parse_declaration"
36914
f1eaae7f962d Bug #1442874: handle "<!>", the empty SGML comment
Georg Brandl <georg@python.org>
parents: 35386
diff changeset
78 if rawdata[j:j+1] == ">":
f1eaae7f962d Bug #1442874: handle "<!>", the empty SGML comment
Georg Brandl <georg@python.org>
parents: 35386
diff changeset
79 # the empty comment <!>
f1eaae7f962d Bug #1442874: handle "<!>", the empty SGML comment
Georg Brandl <georg@python.org>
parents: 35386
diff changeset
80 return j + 1
20065
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
81 if rawdata[j:j+1] in ("-", ""):
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
82 # Start of comment followed by buffer boundary,
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
83 # or just a buffer boundary.
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
84 return -1
28446
500f545a24f0 Patch #545300: Support marked sections.
Martin v. Löwis <martin@v.loewis.de>
parents: 23645
diff changeset
85 # A simple, practical version could look like: ((name|stringlit) S*) + '>'
20065
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
86 n = len(rawdata)
36914
f1eaae7f962d Bug #1442874: handle "<!>", the empty SGML comment
Georg Brandl <georg@python.org>
parents: 35386
diff changeset
87 if rawdata[j:j+2] == '--': #comment
28446
500f545a24f0 Patch #545300: Support marked sections.
Martin v. Löwis <martin@v.loewis.de>
parents: 23645
diff changeset
88 # Locate --.*-- as the body of the comment
500f545a24f0 Patch #545300: Support marked sections.
Martin v. Löwis <martin@v.loewis.de>
parents: 23645
diff changeset
89 return self.parse_comment(i)
500f545a24f0 Patch #545300: Support marked sections.
Martin v. Löwis <martin@v.loewis.de>
parents: 23645
diff changeset
90 elif rawdata[j] == '[': #marked section
500f545a24f0 Patch #545300: Support marked sections.
Martin v. Löwis <martin@v.loewis.de>
parents: 23645
diff changeset
91 # Locate [statusWord [...arbitrary SGML...]] as the body of the marked section
500f545a24f0 Patch #545300: Support marked sections.
Martin v. Löwis <martin@v.loewis.de>
parents: 23645
diff changeset
92 # Where statusWord is one of TEMP, CDATA, IGNORE, INCLUDE, RCDATA
500f545a24f0 Patch #545300: Support marked sections.
Martin v. Löwis <martin@v.loewis.de>
parents: 23645
diff changeset
93 # Note that this is extended by Microsoft Office "Save as Web" function
500f545a24f0 Patch #545300: Support marked sections.
Martin v. Löwis <martin@v.loewis.de>
parents: 23645
diff changeset
94 # to include [if...] and [endif].
500f545a24f0 Patch #545300: Support marked sections.
Martin v. Löwis <martin@v.loewis.de>
parents: 23645
diff changeset
95 return self.parse_marked_section(i)
500f545a24f0 Patch #545300: Support marked sections.
Martin v. Löwis <martin@v.loewis.de>
parents: 23645
diff changeset
96 else: #all other declaration elements
500f545a24f0 Patch #545300: Support marked sections.
Martin v. Löwis <martin@v.loewis.de>
parents: 23645
diff changeset
97 decltype, j = self._scan_name(j, i)
20065
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
98 if j < 0:
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
99 return j
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
100 if decltype == "doctype":
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
101 self._decl_otherchars = ''
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
102 while j < n:
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
103 c = rawdata[j]
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
104 if c == ">":
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
105 # end of declaration syntax
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
106 data = rawdata[i+2:j]
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
107 if decltype == "doctype":
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
108 self.handle_decl(data)
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
109 else:
74070
4ddbb756b602 #13576: add tests about the handling of (possibly broken) condcoms.
Ezio Melotti <ezio.melotti@gmail.com>
parents: 64479
diff changeset
110 # According to the HTML5 specs sections "8.2.4.44 Bogus
4ddbb756b602 #13576: add tests about the handling of (possibly broken) condcoms.
Ezio Melotti <ezio.melotti@gmail.com>
parents: 64479
diff changeset
111 # comment state" and "8.2.4.45 Markup declaration open
4ddbb756b602 #13576: add tests about the handling of (possibly broken) condcoms.
Ezio Melotti <ezio.melotti@gmail.com>
parents: 64479
diff changeset
112 # state", a comment token should be emitted.
4ddbb756b602 #13576: add tests about the handling of (possibly broken) condcoms.
Ezio Melotti <ezio.melotti@gmail.com>
parents: 64479
diff changeset
113 # Calling unknown_decl provides more flexibility though.
20065
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
114 self.unknown_decl(data)
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
115 return j + 1
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
116 if c in "\"'":
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
117 m = _declstringlit_match(rawdata, j)
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
118 if not m:
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
119 return -1 # incomplete
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
120 j = m.end()
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
121 elif c in "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ":
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
122 name, j = self._scan_name(j, i)
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
123 elif c in self._decl_otherchars:
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
124 j = j + 1
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
125 elif c == "[":
28446
500f545a24f0 Patch #545300: Support marked sections.
Martin v. Löwis <martin@v.loewis.de>
parents: 23645
diff changeset
126 # this could be handled in a separate doctype parser
20065
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
127 if decltype == "doctype":
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
128 j = self._parse_doctype_subset(j + 1, i)
64479
8ef68ebec0e1 Take advantage of the frozenset constant optimization.
Raymond Hettinger <python@rcn.com>
parents: 47434
diff changeset
129 elif decltype in {"attlist", "linktype", "link", "element"}:
28446
500f545a24f0 Patch #545300: Support marked sections.
Martin v. Löwis <martin@v.loewis.de>
parents: 23645
diff changeset
130 # must tolerate []'d groups in a content model in an element declaration
500f545a24f0 Patch #545300: Support marked sections.
Martin v. Löwis <martin@v.loewis.de>
parents: 23645
diff changeset
131 # also in data attribute specifications of attlist declaration
500f545a24f0 Patch #545300: Support marked sections.
Martin v. Löwis <martin@v.loewis.de>
parents: 23645
diff changeset
132 # also link type declaration subsets in linktype declarations
500f545a24f0 Patch #545300: Support marked sections.
Martin v. Löwis <martin@v.loewis.de>
parents: 23645
diff changeset
133 # also link attribute specification lists in link declarations
500f545a24f0 Patch #545300: Support marked sections.
Martin v. Löwis <martin@v.loewis.de>
parents: 23645
diff changeset
134 self.error("unsupported '[' char in %s declaration" % decltype)
20065
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
135 else:
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
136 self.error("unexpected '[' char in declaration")
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
137 else:
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
138 self.error(
31515
71a294bd7950 Replace backticks with repr() or "%r"
Walter Dörwald <walter@livinglogic.de>
parents: 28774
diff changeset
139 "unexpected %r char in declaration" % rawdata[j])
20065
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
140 if j < 0:
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
141 return j
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
142 return -1 # incomplete
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
143
28446
500f545a24f0 Patch #545300: Support marked sections.
Martin v. Löwis <martin@v.loewis.de>
parents: 23645
diff changeset
144 # Internal -- parse a marked section
500f545a24f0 Patch #545300: Support marked sections.
Martin v. Löwis <martin@v.loewis.de>
parents: 23645
diff changeset
145 # Override this to handle MS-word extension syntax <![if word]>content<![endif]>
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 36914
diff changeset
146 def parse_marked_section(self, i, report=1):
28446
500f545a24f0 Patch #545300: Support marked sections.
Martin v. Löwis <martin@v.loewis.de>
parents: 23645
diff changeset
147 rawdata= self.rawdata
500f545a24f0 Patch #545300: Support marked sections.
Martin v. Löwis <martin@v.loewis.de>
parents: 23645
diff changeset
148 assert rawdata[i:i+3] == '<![', "unexpected call to parse_marked_section()"
500f545a24f0 Patch #545300: Support marked sections.
Martin v. Löwis <martin@v.loewis.de>
parents: 23645
diff changeset
149 sectName, j = self._scan_name( i+3, i )
500f545a24f0 Patch #545300: Support marked sections.
Martin v. Löwis <martin@v.loewis.de>
parents: 23645
diff changeset
150 if j < 0:
500f545a24f0 Patch #545300: Support marked sections.
Martin v. Löwis <martin@v.loewis.de>
parents: 23645
diff changeset
151 return j
64479
8ef68ebec0e1 Take advantage of the frozenset constant optimization.
Raymond Hettinger <python@rcn.com>
parents: 47434
diff changeset
152 if sectName in {"temp", "cdata", "ignore", "include", "rcdata"}:
28446
500f545a24f0 Patch #545300: Support marked sections.
Martin v. Löwis <martin@v.loewis.de>
parents: 23645
diff changeset
153 # look for standard ]]> ending
500f545a24f0 Patch #545300: Support marked sections.
Martin v. Löwis <martin@v.loewis.de>
parents: 23645
diff changeset
154 match= _markedsectionclose.search(rawdata, i+3)
64479
8ef68ebec0e1 Take advantage of the frozenset constant optimization.
Raymond Hettinger <python@rcn.com>
parents: 47434
diff changeset
155 elif sectName in {"if", "else", "endif"}:
28446
500f545a24f0 Patch #545300: Support marked sections.
Martin v. Löwis <martin@v.loewis.de>
parents: 23645
diff changeset
156 # look for MS Office ]> ending
500f545a24f0 Patch #545300: Support marked sections.
Martin v. Löwis <martin@v.loewis.de>
parents: 23645
diff changeset
157 match= _msmarkedsectionclose.search(rawdata, i+3)
500f545a24f0 Patch #545300: Support marked sections.
Martin v. Löwis <martin@v.loewis.de>
parents: 23645
diff changeset
158 else:
31515
71a294bd7950 Replace backticks with repr() or "%r"
Walter Dörwald <walter@livinglogic.de>
parents: 28774
diff changeset
159 self.error('unknown status keyword %r in marked section' % rawdata[i+3:j])
28446
500f545a24f0 Patch #545300: Support marked sections.
Martin v. Löwis <martin@v.loewis.de>
parents: 23645
diff changeset
160 if not match:
500f545a24f0 Patch #545300: Support marked sections.
Martin v. Löwis <martin@v.loewis.de>
parents: 23645
diff changeset
161 return -1
500f545a24f0 Patch #545300: Support marked sections.
Martin v. Löwis <martin@v.loewis.de>
parents: 23645
diff changeset
162 if report:
500f545a24f0 Patch #545300: Support marked sections.
Martin v. Löwis <martin@v.loewis.de>
parents: 23645
diff changeset
163 j = match.start(0)
500f545a24f0 Patch #545300: Support marked sections.
Martin v. Löwis <martin@v.loewis.de>
parents: 23645
diff changeset
164 self.unknown_decl(rawdata[i+3: j])
500f545a24f0 Patch #545300: Support marked sections.
Martin v. Löwis <martin@v.loewis.de>
parents: 23645
diff changeset
165 return match.end(0)
28774
c9bee11eaa8b Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 28446
diff changeset
166
28446
500f545a24f0 Patch #545300: Support marked sections.
Martin v. Löwis <martin@v.loewis.de>
parents: 23645
diff changeset
167 # Internal -- parse comment, return length or -1 if not terminated
500f545a24f0 Patch #545300: Support marked sections.
Martin v. Löwis <martin@v.loewis.de>
parents: 23645
diff changeset
168 def parse_comment(self, i, report=1):
500f545a24f0 Patch #545300: Support marked sections.
Martin v. Löwis <martin@v.loewis.de>
parents: 23645
diff changeset
169 rawdata = self.rawdata
500f545a24f0 Patch #545300: Support marked sections.
Martin v. Löwis <martin@v.loewis.de>
parents: 23645
diff changeset
170 if rawdata[i:i+4] != '<!--':
500f545a24f0 Patch #545300: Support marked sections.
Martin v. Löwis <martin@v.loewis.de>
parents: 23645
diff changeset
171 self.error('unexpected call to parse_comment()')
500f545a24f0 Patch #545300: Support marked sections.
Martin v. Löwis <martin@v.loewis.de>
parents: 23645
diff changeset
172 match = _commentclose.search(rawdata, i+4)
500f545a24f0 Patch #545300: Support marked sections.
Martin v. Löwis <martin@v.loewis.de>
parents: 23645
diff changeset
173 if not match:
500f545a24f0 Patch #545300: Support marked sections.
Martin v. Löwis <martin@v.loewis.de>
parents: 23645
diff changeset
174 return -1
500f545a24f0 Patch #545300: Support marked sections.
Martin v. Löwis <martin@v.loewis.de>
parents: 23645
diff changeset
175 if report:
500f545a24f0 Patch #545300: Support marked sections.
Martin v. Löwis <martin@v.loewis.de>
parents: 23645
diff changeset
176 j = match.start(0)
500f545a24f0 Patch #545300: Support marked sections.
Martin v. Löwis <martin@v.loewis.de>
parents: 23645
diff changeset
177 self.handle_comment(rawdata[i+4: j])
500f545a24f0 Patch #545300: Support marked sections.
Martin v. Löwis <martin@v.loewis.de>
parents: 23645
diff changeset
178 return match.end(0)
500f545a24f0 Patch #545300: Support marked sections.
Martin v. Löwis <martin@v.loewis.de>
parents: 23645
diff changeset
179
20065
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
180 # Internal -- scan past the internal subset in a <!DOCTYPE declaration,
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
181 # returning the index just past any whitespace following the trailing ']'.
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
182 def _parse_doctype_subset(self, i, declstartpos):
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
183 rawdata = self.rawdata
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
184 n = len(rawdata)
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
185 j = i
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
186 while j < n:
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
187 c = rawdata[j]
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
188 if c == "<":
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
189 s = rawdata[j:j+2]
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
190 if s == "<":
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
191 # end of buffer; incomplete
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
192 return -1
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
193 if s != "<!":
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
194 self.updatepos(declstartpos, j + 1)
31515
71a294bd7950 Replace backticks with repr() or "%r"
Walter Dörwald <walter@livinglogic.de>
parents: 28774
diff changeset
195 self.error("unexpected char in internal subset (in %r)" % s)
20065
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
196 if (j + 2) == n:
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
197 # end of buffer; incomplete
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
198 return -1
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
199 if (j + 4) > n:
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
200 # end of buffer; incomplete
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
201 return -1
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
202 if rawdata[j:j+4] == "<!--":
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
203 j = self.parse_comment(j, report=0)
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
204 if j < 0:
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
205 return j
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
206 continue
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
207 name, j = self._scan_name(j + 2, declstartpos)
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
208 if j == -1:
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
209 return -1
64479
8ef68ebec0e1 Take advantage of the frozenset constant optimization.
Raymond Hettinger <python@rcn.com>
parents: 47434
diff changeset
210 if name not in {"attlist", "element", "entity", "notation"}:
20065
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
211 self.updatepos(declstartpos, j + 2)
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
212 self.error(
31515
71a294bd7950 Replace backticks with repr() or "%r"
Walter Dörwald <walter@livinglogic.de>
parents: 28774
diff changeset
213 "unknown declaration %r in internal subset" % name)
20065
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
214 # handle the individual names
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
215 meth = getattr(self, "_parse_doctype_" + name)
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
216 j = meth(j, declstartpos)
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
217 if j < 0:
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
218 return j
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
219 elif c == "%":
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
220 # parameter entity reference
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
221 if (j + 1) == n:
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
222 # end of buffer; incomplete
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
223 return -1
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
224 s, j = self._scan_name(j + 1, declstartpos)
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
225 if j < 0:
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
226 return j
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
227 if rawdata[j] == ";":
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
228 j = j + 1
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
229 elif c == "]":
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
230 j = j + 1
23645
e1e90b0f1f45 Remove uses of the string and types modules:
Walter Dörwald <walter@livinglogic.de>
parents: 23590
diff changeset
231 while j < n and rawdata[j].isspace():
20065
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
232 j = j + 1
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
233 if j < n:
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
234 if rawdata[j] == ">":
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
235 return j
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
236 self.updatepos(declstartpos, j)
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
237 self.error("unexpected char after internal subset")
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
238 else:
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
239 return -1
23645
e1e90b0f1f45 Remove uses of the string and types modules:
Walter Dörwald <walter@livinglogic.de>
parents: 23590
diff changeset
240 elif c.isspace():
20065
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
241 j = j + 1
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
242 else:
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
243 self.updatepos(declstartpos, j)
31515
71a294bd7950 Replace backticks with repr() or "%r"
Walter Dörwald <walter@livinglogic.de>
parents: 28774
diff changeset
244 self.error("unexpected char %r in internal subset" % c)
20065
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
245 # end of buffer reached
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
246 return -1
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
247
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
248 # Internal -- scan past <!ELEMENT declarations
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
249 def _parse_doctype_element(self, i, declstartpos):
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
250 name, j = self._scan_name(i, declstartpos)
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
251 if j == -1:
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
252 return -1
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
253 # style content model; just skip until '>'
20699
fa0bf6c79e04 Re-arrange things and remove some unused variables/imports to keep pychecker
Fred Drake <fdrake@acm.org>
parents: 20416
diff changeset
254 rawdata = self.rawdata
20065
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
255 if '>' in rawdata[j:]:
23590
a4bb18212b48 Use string methods where possible, and remove import string
Neal Norwitz <nnorwitz@gmail.com>
parents: 20699
diff changeset
256 return rawdata.find(">", j) + 1
20065
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
257 return -1
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
258
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
259 # Internal -- scan past <!ATTLIST declarations
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
260 def _parse_doctype_attlist(self, i, declstartpos):
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
261 rawdata = self.rawdata
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
262 name, j = self._scan_name(i, declstartpos)
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
263 c = rawdata[j:j+1]
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
264 if c == "":
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
265 return -1
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
266 if c == ">":
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
267 return j + 1
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
268 while 1:
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
269 # scan a series of attribute descriptions; simplified:
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
270 # name type [value] [#constraint]
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
271 name, j = self._scan_name(j, declstartpos)
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
272 if j < 0:
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
273 return j
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
274 c = rawdata[j:j+1]
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
275 if c == "":
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
276 return -1
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
277 if c == "(":
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
278 # an enumerated type; look for ')'
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
279 if ")" in rawdata[j:]:
23590
a4bb18212b48 Use string methods where possible, and remove import string
Neal Norwitz <nnorwitz@gmail.com>
parents: 20699
diff changeset
280 j = rawdata.find(")", j) + 1
20065
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
281 else:
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
282 return -1
23645
e1e90b0f1f45 Remove uses of the string and types modules:
Walter Dörwald <walter@livinglogic.de>
parents: 23590
diff changeset
283 while rawdata[j:j+1].isspace():
20065
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
284 j = j + 1
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
285 if not rawdata[j:]:
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
286 # end of buffer, incomplete
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
287 return -1
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
288 else:
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
289 name, j = self._scan_name(j, declstartpos)
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
290 c = rawdata[j:j+1]
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
291 if not c:
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
292 return -1
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
293 if c in "'\"":
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
294 m = _declstringlit_match(rawdata, j)
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
295 if m:
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
296 j = m.end()
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
297 else:
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
298 return -1
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
299 c = rawdata[j:j+1]
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
300 if not c:
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
301 return -1
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
302 if c == "#":
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
303 if rawdata[j:] == "#":
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
304 # end of buffer
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
305 return -1
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
306 name, j = self._scan_name(j + 1, declstartpos)
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
307 if j < 0:
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
308 return j
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
309 c = rawdata[j:j+1]
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
310 if not c:
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
311 return -1
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
312 if c == '>':
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
313 # all done
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
314 return j + 1
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
315
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
316 # Internal -- scan past <!NOTATION declarations
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
317 def _parse_doctype_notation(self, i, declstartpos):
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
318 name, j = self._scan_name(i, declstartpos)
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
319 if j < 0:
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
320 return j
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
321 rawdata = self.rawdata
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
322 while 1:
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
323 c = rawdata[j:j+1]
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
324 if not c:
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
325 # end of buffer; incomplete
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
326 return -1
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
327 if c == '>':
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
328 return j + 1
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
329 if c in "'\"":
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
330 m = _declstringlit_match(rawdata, j)
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
331 if not m:
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
332 return -1
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
333 j = m.end()
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
334 else:
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
335 name, j = self._scan_name(j, declstartpos)
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
336 if j < 0:
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
337 return j
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
338
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
339 # Internal -- scan past <!ENTITY declarations
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
340 def _parse_doctype_entity(self, i, declstartpos):
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
341 rawdata = self.rawdata
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
342 if rawdata[i:i+1] == "%":
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
343 j = i + 1
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
344 while 1:
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
345 c = rawdata[j:j+1]
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
346 if not c:
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
347 return -1
23645
e1e90b0f1f45 Remove uses of the string and types modules:
Walter Dörwald <walter@livinglogic.de>
parents: 23590
diff changeset
348 if c.isspace():
20065
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
349 j = j + 1
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
350 else:
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
351 break
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
352 else:
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
353 j = i
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
354 name, j = self._scan_name(j, declstartpos)
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
355 if j < 0:
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
356 return j
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
357 while 1:
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
358 c = self.rawdata[j:j+1]
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
359 if not c:
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
360 return -1
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
361 if c in "'\"":
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
362 m = _declstringlit_match(rawdata, j)
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
363 if m:
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
364 j = m.end()
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
365 else:
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
366 return -1 # incomplete
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
367 elif c == ">":
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
368 return j + 1
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
369 else:
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
370 name, j = self._scan_name(j, declstartpos)
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
371 if j < 0:
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
372 return j
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
373
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
374 # Internal -- scan a name token and the new position and the token, or
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
375 # return -1 if we've reached the end of the buffer.
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
376 def _scan_name(self, i, declstartpos):
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
377 rawdata = self.rawdata
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
378 n = len(rawdata)
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
379 if i == n:
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
380 return None, -1
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
381 m = _declname_match(rawdata, i)
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
382 if m:
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
383 s = m.group()
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
384 name = s.strip()
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
385 if (i + len(s)) == n:
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
386 return None, -1 # end of buffer
23590
a4bb18212b48 Use string methods where possible, and remove import string
Neal Norwitz <nnorwitz@gmail.com>
parents: 20699
diff changeset
387 return name.lower(), m.end()
20065
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
388 else:
f84691b11402 New base class for the SGMLParser and HTMLParser classes from the sgmllib
Fred Drake <fdrake@acm.org>
parents:
diff changeset
389 self.updatepos(declstartpos, i)
32603
799bcdff2cdc [Patch #965175] Incorporate a suggestion for a better error message
Andrew M. Kuchling <amk@amk.ca>
parents: 31515
diff changeset
390 self.error("expected name token at %r"
799bcdff2cdc [Patch #965175] Incorporate a suggestion for a better error message
Andrew M. Kuchling <amk@amk.ca>
parents: 31515
diff changeset
391 % rawdata[declstartpos:declstartpos+20])
20699
fa0bf6c79e04 Re-arrange things and remove some unused variables/imports to keep pychecker
Fred Drake <fdrake@acm.org>
parents: 20416
diff changeset
392
fa0bf6c79e04 Re-arrange things and remove some unused variables/imports to keep pychecker
Fred Drake <fdrake@acm.org>
parents: 20416
diff changeset
393 # To be overridden -- handlers for unknown objects
fa0bf6c79e04 Re-arrange things and remove some unused variables/imports to keep pychecker
Fred Drake <fdrake@acm.org>
parents: 20416
diff changeset
394 def unknown_decl(self, data):
fa0bf6c79e04 Re-arrange things and remove some unused variables/imports to keep pychecker
Fred Drake <fdrake@acm.org>
parents: 20416
diff changeset
395 pass