cpython-lru_cache-weakref: Lib/sre

annotate Lib/sre_parse.py @ 78069:a8aa77458e12 3.2

Add c_size_t and c_ssize_t to the ctypes summary table.

author	Antoine Pitrou <solipsis@pitrou.net>
date	Thu, 12 Jul 2012 20:31:50 +0200
parents	9258f405a7ac
children	29983dd7191e be6c38d1817b

rev	line source
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	1 #
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	2 # Secret Labs' Regular Expression Engine
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	3 #
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	4 # convert re-style regular expression to sre pattern
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	5 #
15855 152d00718006 SRE fixes for 2.1 alpha: Fredrik Lundh <fredrik@pythonware.com> parents: 15355 diff changeset	6 # Copyright (c) 1998-2001 by Secret Labs AB. All rights reserved.
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	7 #
13789 4edd13eb759d SRE 0.9.8: passes the entire test suite Fredrik Lundh <fredrik@pythonware.com> parents: 13677 diff changeset	8 # See the sre.py file for information on usage and redistribution.
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	9 #
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	10
19693 617f5e72b0ec Added docstrings by Neal Norwitz. This closes SF bug #450980. Fred Drake <fdrake@acm.org> parents: 17144 diff changeset	11 """Internal support module for sre"""
617f5e72b0ec Added docstrings by Neal Norwitz. This closes SF bug #450980. Fred Drake <fdrake@acm.org> parents: 17144 diff changeset	12
15858 81e3d8626adf SRE: stricter pattern syntax checking (covers parts of bug #115900) Fredrik Lundh <fredrik@pythonware.com> parents: 15855 diff changeset	13 # XXX: show string offset and offending character for all errors
81e3d8626adf SRE: stricter pattern syntax checking (covers parts of bug #115900) Fredrik Lundh <fredrik@pythonware.com> parents: 15855 diff changeset	14
33257 11a4fbc88a80 PEP 292 classes Template and SafeTemplate are added to the string module. Barry Warsaw <barry@python.org> parents: 31829 diff changeset	15 import sys
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	16
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	17 from sre_constants import *
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	18
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	19 SPECIAL_CHARS = ".\\[{()*+?^$\|"
14379 b069d7d9ce8a -- tightened up parsing of octal numbers Fredrik Lundh <fredrik@pythonware.com> parents: 14376 diff changeset	20 REPEAT_CHARS = "*+?{"
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	21
34635 5f26298811b7 Complete the previous effort to factor out constant expressions Raymond Hettinger <python@rcn.com> parents: 33400 diff changeset	22 DIGITS = set("0123456789")
12339 9e9666f679f7 Fredrik Lundh: new snapshot. Mostly reindented. Guido van Rossum <guido@python.org> parents: 12213 diff changeset	23
34635 5f26298811b7 Complete the previous effort to factor out constant expressions Raymond Hettinger <python@rcn.com> parents: 33400 diff changeset	24 OCTDIGITS = set("01234567")
5f26298811b7 Complete the previous effort to factor out constant expressions Raymond Hettinger <python@rcn.com> parents: 33400 diff changeset	25 HEXDIGITS = set("0123456789abcdefABCDEF")
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	26
34635 5f26298811b7 Complete the previous effort to factor out constant expressions Raymond Hettinger <python@rcn.com> parents: 33400 diff changeset	27 WHITESPACE = set(" \t\n\r\v\f")
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	28
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	29 ESCAPES = {
16663 94a38443f99b - restored 1.5.2 compatibility (sorry, eric) Fredrik Lundh <fredrik@pythonware.com> parents: 16613 diff changeset	30 r"\a": (LITERAL, ord("\a")),
94a38443f99b - restored 1.5.2 compatibility (sorry, eric) Fredrik Lundh <fredrik@pythonware.com> parents: 16613 diff changeset	31 r"\b": (LITERAL, ord("\b")),
94a38443f99b - restored 1.5.2 compatibility (sorry, eric) Fredrik Lundh <fredrik@pythonware.com> parents: 16613 diff changeset	32 r"\f": (LITERAL, ord("\f")),
94a38443f99b - restored 1.5.2 compatibility (sorry, eric) Fredrik Lundh <fredrik@pythonware.com> parents: 16613 diff changeset	33 r"\n": (LITERAL, ord("\n")),
94a38443f99b - restored 1.5.2 compatibility (sorry, eric) Fredrik Lundh <fredrik@pythonware.com> parents: 16613 diff changeset	34 r"\r": (LITERAL, ord("\r")),
94a38443f99b - restored 1.5.2 compatibility (sorry, eric) Fredrik Lundh <fredrik@pythonware.com> parents: 16613 diff changeset	35 r"\t": (LITERAL, ord("\t")),
94a38443f99b - restored 1.5.2 compatibility (sorry, eric) Fredrik Lundh <fredrik@pythonware.com> parents: 16613 diff changeset	36 r"\v": (LITERAL, ord("\v")),
13201 a093fad92ccf the mad patcher strikes again: Fredrik Lundh <fredrik@pythonware.com> parents: 13198 diff changeset	37 r"\\": (LITERAL, ord("\\"))
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	38 }
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	39
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	40 CATEGORIES = {
15855 152d00718006 SRE fixes for 2.1 alpha: Fredrik Lundh <fredrik@pythonware.com> parents: 15355 diff changeset	41 r"\A": (AT, AT_BEGINNING_STRING), # start of string
13166 023a2018b96d - fixed split behaviour on empty matches Fredrik Lundh <fredrik@pythonware.com> parents: 13139 diff changeset	42 r"\b": (AT, AT_BOUNDARY),
023a2018b96d - fixed split behaviour on empty matches Fredrik Lundh <fredrik@pythonware.com> parents: 13139 diff changeset	43 r"\B": (AT, AT_NON_BOUNDARY),
023a2018b96d - fixed split behaviour on empty matches Fredrik Lundh <fredrik@pythonware.com> parents: 13139 diff changeset	44 r"\d": (IN, [(CATEGORY, CATEGORY_DIGIT)]),
023a2018b96d - fixed split behaviour on empty matches Fredrik Lundh <fredrik@pythonware.com> parents: 13139 diff changeset	45 r"\D": (IN, [(CATEGORY, CATEGORY_NOT_DIGIT)]),
023a2018b96d - fixed split behaviour on empty matches Fredrik Lundh <fredrik@pythonware.com> parents: 13139 diff changeset	46 r"\s": (IN, [(CATEGORY, CATEGORY_SPACE)]),
023a2018b96d - fixed split behaviour on empty matches Fredrik Lundh <fredrik@pythonware.com> parents: 13139 diff changeset	47 r"\S": (IN, [(CATEGORY, CATEGORY_NOT_SPACE)]),
023a2018b96d - fixed split behaviour on empty matches Fredrik Lundh <fredrik@pythonware.com> parents: 13139 diff changeset	48 r"\w": (IN, [(CATEGORY, CATEGORY_WORD)]),
023a2018b96d - fixed split behaviour on empty matches Fredrik Lundh <fredrik@pythonware.com> parents: 13139 diff changeset	49 r"\W": (IN, [(CATEGORY, CATEGORY_NOT_WORD)]),
15855 152d00718006 SRE fixes for 2.1 alpha: Fredrik Lundh <fredrik@pythonware.com> parents: 15355 diff changeset	50 r"\Z": (AT, AT_END_STRING), # end of string
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	51 }
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	52
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	53 FLAGS = {
13118 7ff40da08c2a towards 1.6b1 Fredrik Lundh <fredrik@pythonware.com> parents: 12903 diff changeset	54 # standard flags
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	55 "i": SRE_FLAG_IGNORECASE,
9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	56 "L": SRE_FLAG_LOCALE,
9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	57 "m": SRE_FLAG_MULTILINE,
9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	58 "s": SRE_FLAG_DOTALL,
13118 7ff40da08c2a towards 1.6b1 Fredrik Lundh <fredrik@pythonware.com> parents: 12903 diff changeset	59 "x": SRE_FLAG_VERBOSE,
7ff40da08c2a towards 1.6b1 Fredrik Lundh <fredrik@pythonware.com> parents: 12903 diff changeset	60 # extensions
48769 83b2d07b475b #2834: Change re module semantics, so that str and bytes mixing is forbidden, Antoine Pitrou <solipsis@pitrou.net> parents: 48554 diff changeset	61 "a": SRE_FLAG_ASCII,
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	62 "t": SRE_FLAG_TEMPLATE,
13118 7ff40da08c2a towards 1.6b1 Fredrik Lundh <fredrik@pythonware.com> parents: 12903 diff changeset	63 "u": SRE_FLAG_UNICODE,
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	64 }
9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	65
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	66 class Pattern:
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	67 # master pattern object. keeps track of global attributes
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	68 def __init__(self):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	69 self.flags = 0
15355 2a3696d32721 -- properly reset groups in findall (bug #117612) Fredrik Lundh <fredrik@pythonware.com> parents: 15091 diff changeset	70 self.open = []
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	71 self.groups = 1
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	72 self.groupdict = {}
15355 2a3696d32721 -- properly reset groups in findall (bug #117612) Fredrik Lundh <fredrik@pythonware.com> parents: 15091 diff changeset	73 def opengroup(self, name=None):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	74 gid = self.groups
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	75 self.groups = gid + 1
23627 a6a765c7d33b Replace boolean test with is None. Raymond Hettinger <python@rcn.com> parents: 23620 diff changeset	76 if name is not None:
20860 0caea5fefa5c Improved error msg when a symbolic group name is redefined. Added docs Tim Peters <tim.peters@gmail.com> parents: 20830 diff changeset	77 ogid = self.groupdict.get(name, None)
0caea5fefa5c Improved error msg when a symbolic group name is redefined. Added docs Tim Peters <tim.peters@gmail.com> parents: 20830 diff changeset	78 if ogid is not None:
43020 907a45df1295 Raise statement normalization in Lib/. Collin Winter <collinw@gmail.com> parents: 41986 diff changeset	79 raise error("redefinition of group name %s as group %d; "
907a45df1295 Raise statement normalization in Lib/. Collin Winter <collinw@gmail.com> parents: 41986 diff changeset	80 "was group %d" % (repr(name), gid, ogid))
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	81 self.groupdict[name] = gid
15355 2a3696d32721 -- properly reset groups in findall (bug #117612) Fredrik Lundh <fredrik@pythonware.com> parents: 15091 diff changeset	82 self.open.append(gid)
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	83 return gid
15355 2a3696d32721 -- properly reset groups in findall (bug #117612) Fredrik Lundh <fredrik@pythonware.com> parents: 15091 diff changeset	84 def closegroup(self, gid):
2a3696d32721 -- properly reset groups in findall (bug #117612) Fredrik Lundh <fredrik@pythonware.com> parents: 15091 diff changeset	85 self.open.remove(gid)
2a3696d32721 -- properly reset groups in findall (bug #117612) Fredrik Lundh <fredrik@pythonware.com> parents: 15091 diff changeset	86 def checkgroup(self, gid):
2a3696d32721 -- properly reset groups in findall (bug #117612) Fredrik Lundh <fredrik@pythonware.com> parents: 15091 diff changeset	87 return gid < self.groups and gid not in self.open
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	88
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	89 class SubPattern:
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	90 # a subpattern, in intermediate form
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	91 def __init__(self, pattern, data=None):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	92 self.pattern = pattern
23627 a6a765c7d33b Replace boolean test with is None. Raymond Hettinger <python@rcn.com> parents: 23620 diff changeset	93 if data is None:
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	94 data = []
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	95 self.data = data
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	96 self.width = None
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	97 def dump(self, level=0):
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	98 nl = 1
41986 6464cc638faa Merged revisions 55795-55816 via svnmerge from Guido van Rossum <guido@python.org> parents: 41585 diff changeset	99 seqtypes = (tuple, list)
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	100 for op, av in self.data:
40775 4cdf21a0dbbc Fix most trivially-findable print statements. Guido van Rossum <guido@python.org> parents: 40644 diff changeset	101 print(level*" " + op, end=' '); nl = 0
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	102 if op == "in":
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	103 # member sublanguage
40775 4cdf21a0dbbc Fix most trivially-findable print statements. Guido van Rossum <guido@python.org> parents: 40644 diff changeset	104 print(); nl = 1
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	105 for op, a in av:
40775 4cdf21a0dbbc Fix most trivially-findable print statements. Guido van Rossum <guido@python.org> parents: 40644 diff changeset	106 print((level+1)*" " + op, a)
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	107 elif op == "branch":
40775 4cdf21a0dbbc Fix most trivially-findable print statements. Guido van Rossum <guido@python.org> parents: 40644 diff changeset	108 print(); nl = 1
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	109 i = 0
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	110 for a in av[1]:
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	111 if i > 0:
40775 4cdf21a0dbbc Fix most trivially-findable print statements. Guido van Rossum <guido@python.org> parents: 40644 diff changeset	112 print(level*" " + "or")
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	113 a.dump(level+1); nl = 1
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	114 i = i + 1
41986 6464cc638faa Merged revisions 55795-55816 via svnmerge from Guido van Rossum <guido@python.org> parents: 41585 diff changeset	115 elif isinstance(av, seqtypes):
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	116 for a in av:
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	117 if isinstance(a, SubPattern):
40775 4cdf21a0dbbc Fix most trivially-findable print statements. Guido van Rossum <guido@python.org> parents: 40644 diff changeset	118 if not nl: print()
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	119 a.dump(level+1); nl = 1
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	120 else:
40775 4cdf21a0dbbc Fix most trivially-findable print statements. Guido van Rossum <guido@python.org> parents: 40644 diff changeset	121 print(a, end=' ') ; nl = 0
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	122 else:
40775 4cdf21a0dbbc Fix most trivially-findable print statements. Guido van Rossum <guido@python.org> parents: 40644 diff changeset	123 print(av, end=' ') ; nl = 0
4cdf21a0dbbc Fix most trivially-findable print statements. Guido van Rossum <guido@python.org> parents: 40644 diff changeset	124 if not nl: print()
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	125 def __repr__(self):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	126 return repr(self.data)
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	127 def __len__(self):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	128 return len(self.data)
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	129 def __delitem__(self, index):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	130 del self.data[index]
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	131 def __getitem__(self, index):
40582 67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 35544 diff changeset	132 if isinstance(index, slice):
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 35544 diff changeset	133 return SubPattern(self.pattern, self.data[index])
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	134 return self.data[index]
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	135 def __setitem__(self, index, code):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	136 self.data[index] = code
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	137 def insert(self, index, code):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	138 self.data.insert(index, code)
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	139 def append(self, code):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	140 self.data.append(code)
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	141 def getwidth(self):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	142 # determine the width (min, max) for this subpattern
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	143 if self.width:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	144 return self.width
40644 1c26505c1de6 Rip out 'long' and 'L'-suffixed integer literals. Guido van Rossum <guido@python.org> parents: 40582 diff changeset	145 lo = hi = 0
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	146 UNITCODES = (ANY, RANGE, IN, LITERAL, NOT_LITERAL, CATEGORY)
a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	147 REPEATCODES = (MIN_REPEAT, MAX_REPEAT)
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	148 for op, av in self.data:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	149 if op is BRANCH:
44160 9d152530910a Removed PyInt_GetMax and sys.maxint Christian Heimes <christian@cheimes.de> parents: 43777 diff changeset	150 i = sys.maxsize
13793 4c10bc811fed -- fixed width calculations for alternations Fredrik Lundh <fredrik@pythonware.com> parents: 13789 diff changeset	151 j = 0
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	152 for av in av[1]:
13793 4c10bc811fed -- fixed width calculations for alternations Fredrik Lundh <fredrik@pythonware.com> parents: 13789 diff changeset	153 l, h = av.getwidth()
4c10bc811fed -- fixed width calculations for alternations Fredrik Lundh <fredrik@pythonware.com> parents: 13789 diff changeset	154 i = min(i, l)
13795 8295f8506eae final 0.9.8 updates: Fredrik Lundh <fredrik@pythonware.com> parents: 13793 diff changeset	155 j = max(j, h)
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	156 lo = lo + i
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	157 hi = hi + j
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	158 elif op is CALL:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	159 i, j = av.getwidth()
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	160 lo = lo + i
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	161 hi = hi + j
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	162 elif op is SUBPATTERN:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	163 i, j = av[1].getwidth()
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	164 lo = lo + i
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	165 hi = hi + j
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	166 elif op in REPEATCODES:
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	167 i, j = av[2].getwidth()
40644 1c26505c1de6 Rip out 'long' and 'L'-suffixed integer literals. Guido van Rossum <guido@python.org> parents: 40582 diff changeset	168 lo = lo + int(i) * av[0]
1c26505c1de6 Rip out 'long' and 'L'-suffixed integer literals. Guido van Rossum <guido@python.org> parents: 40582 diff changeset	169 hi = hi + int(j) * av[1]
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	170 elif op in UNITCODES:
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	171 lo = lo + 1
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	172 hi = hi + 1
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	173 elif op == SUCCESS:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	174 break
44160 9d152530910a Removed PyInt_GetMax and sys.maxint Christian Heimes <christian@cheimes.de> parents: 43777 diff changeset	175 self.width = int(min(lo, sys.maxsize)), int(min(hi, sys.maxsize))
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	176 return self.width
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	177
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	178 class Tokenizer:
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	179 def __init__(self, string):
13305 33ce63b06359 - fixed "{ in any other context" bug Fredrik Lundh <fredrik@pythonware.com> parents: 13299 diff changeset	180 self.string = string
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	181 self.index = 0
13305 33ce63b06359 - fixed "{ in any other context" bug Fredrik Lundh <fredrik@pythonware.com> parents: 13299 diff changeset	182 self.__next()
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	183 def __next(self):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	184 if self.index >= len(self.string):
13305 33ce63b06359 - fixed "{ in any other context" bug Fredrik Lundh <fredrik@pythonware.com> parents: 13299 diff changeset	185 self.next = None
33ce63b06359 - fixed "{ in any other context" bug Fredrik Lundh <fredrik@pythonware.com> parents: 13299 diff changeset	186 return
43604 b18f6092f6e0 Patch 1280, by Alexandre Vassalotti. Guido van Rossum <guido@python.org> parents: 43100 diff changeset	187 char = self.string[self.index:self.index+1]
b18f6092f6e0 Patch 1280, by Alexandre Vassalotti. Guido van Rossum <guido@python.org> parents: 43100 diff changeset	188 # Special case for the str8, since indexing returns a integer
b18f6092f6e0 Patch 1280, by Alexandre Vassalotti. Guido van Rossum <guido@python.org> parents: 43100 diff changeset	189 # XXX This is only needed for test_bug_926075 in test_re.py
45874 83c55acfb959 Fix 're' to work on bytes. It could do with a few more tests, though. Thomas Wouters <thomas@python.org> parents: 44160 diff changeset	190 if char and isinstance(char, bytes):
83c55acfb959 Fix 're' to work on bytes. It could do with a few more tests, though. Thomas Wouters <thomas@python.org> parents: 44160 diff changeset	191 char = chr(char[0])
43604 b18f6092f6e0 Patch 1280, by Alexandre Vassalotti. Guido van Rossum <guido@python.org> parents: 43100 diff changeset	192 if char == "\\":
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	193 try:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	194 c = self.string[self.index + 1]
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	195 except IndexError:
43020 907a45df1295 Raise statement normalization in Lib/. Collin Winter <collinw@gmail.com> parents: 41986 diff changeset	196 raise error("bogus escape (end of line)")
43777 0d462d789b18 Merging the py3k-pep3137 branch back into the py3k branch. Guido van Rossum <guido@python.org> parents: 43604 diff changeset	197 if isinstance(self.string, bytes):
48554 1a2f8dde81ec #3231: re.compile fails with some bytes patterns Antoine Pitrou <solipsis@pitrou.net> parents: 45874 diff changeset	198 c = chr(c)
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	199 char = char + c
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	200 self.index = self.index + len(char)
13305 33ce63b06359 - fixed "{ in any other context" bug Fredrik Lundh <fredrik@pythonware.com> parents: 13299 diff changeset	201 self.next = char
14316 511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	202 def match(self, char, skip=1):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	203 if char == self.next:
14316 511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	204 if skip:
511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	205 self.__next()
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	206 return 1
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	207 return 0
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	208 def get(self):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	209 this = self.next
13305 33ce63b06359 - fixed "{ in any other context" bug Fredrik Lundh <fredrik@pythonware.com> parents: 13299 diff changeset	210 self.__next()
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	211 return this
13305 33ce63b06359 - fixed "{ in any other context" bug Fredrik Lundh <fredrik@pythonware.com> parents: 13299 diff changeset	212 def tell(self):
33ce63b06359 - fixed "{ in any other context" bug Fredrik Lundh <fredrik@pythonware.com> parents: 13299 diff changeset	213 return self.index, self.next
33ce63b06359 - fixed "{ in any other context" bug Fredrik Lundh <fredrik@pythonware.com> parents: 13299 diff changeset	214 def seek(self, index):
33ce63b06359 - fixed "{ in any other context" bug Fredrik Lundh <fredrik@pythonware.com> parents: 13299 diff changeset	215 self.index, self.next = index
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	216
13122 53d00e7c5848 - make sure group names are valid identifiers Fredrik Lundh <fredrik@pythonware.com> parents: 13121 diff changeset	217 def isident(char):
53d00e7c5848 - make sure group names are valid identifiers Fredrik Lundh <fredrik@pythonware.com> parents: 13121 diff changeset	218 return "a" <= char <= "z" or "A" <= char <= "Z" or char == "_"
53d00e7c5848 - make sure group names are valid identifiers Fredrik Lundh <fredrik@pythonware.com> parents: 13121 diff changeset	219
53d00e7c5848 - make sure group names are valid identifiers Fredrik Lundh <fredrik@pythonware.com> parents: 13121 diff changeset	220 def isdigit(char):
53d00e7c5848 - make sure group names are valid identifiers Fredrik Lundh <fredrik@pythonware.com> parents: 13121 diff changeset	221 return "0" <= char <= "9"
53d00e7c5848 - make sure group names are valid identifiers Fredrik Lundh <fredrik@pythonware.com> parents: 13121 diff changeset	222
53d00e7c5848 - make sure group names are valid identifiers Fredrik Lundh <fredrik@pythonware.com> parents: 13121 diff changeset	223 def isname(name):
53d00e7c5848 - make sure group names are valid identifiers Fredrik Lundh <fredrik@pythonware.com> parents: 13121 diff changeset	224 # check that group name is a valid string
53d00e7c5848 - make sure group names are valid identifiers Fredrik Lundh <fredrik@pythonware.com> parents: 13121 diff changeset	225 if not isident(name[0]):
22867 e2f2187c3d2c Partial introduction of bools where appropriate. Guido van Rossum <guido@python.org> parents: 21925 diff changeset	226 return False
33397 98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	227 for char in name[1:]:
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	228 if not isident(char) and not isdigit(char):
22867 e2f2187c3d2c Partial introduction of bools where appropriate. Guido van Rossum <guido@python.org> parents: 21925 diff changeset	229 return False
e2f2187c3d2c Partial introduction of bools where appropriate. Guido van Rossum <guido@python.org> parents: 21925 diff changeset	230 return True
13122 53d00e7c5848 - make sure group names are valid identifiers Fredrik Lundh <fredrik@pythonware.com> parents: 13121 diff changeset	231
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	232 def _class_escape(source, escape):
9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	233 # handle escape code inside character class
9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	234 code = ESCAPES.get(escape)
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	235 if code:
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	236 return code
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	237 code = CATEGORIES.get(escape)
9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	238 if code:
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	239 return code
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	240 try:
33397 98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	241 c = escape[1:2]
98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	242 if c == "x":
14316 511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	243 # hexadecimal escape (exactly two digits)
511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	244 while source.next in HEXDIGITS and len(escape) < 4:
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	245 escape = escape + source.get()
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	246 escape = escape[2:]
14316 511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	247 if len(escape) != 2:
43020 907a45df1295 Raise statement normalization in Lib/. Collin Winter <collinw@gmail.com> parents: 41986 diff changeset	248 raise error("bogus escape: %s" % repr("\\" + escape))
33257 11a4fbc88a80 PEP 292 classes Template and SafeTemplate are added to the string module. Barry Warsaw <barry@python.org> parents: 31829 diff changeset	249 return LITERAL, int(escape, 16) & 0xff
33397 98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	250 elif c in OCTDIGITS:
14316 511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	251 # octal escape (up to three digits)
33397 98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	252 while source.next in OCTDIGITS and len(escape) < 4:
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	253 escape = escape + source.get()
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	254 escape = escape[1:]
33257 11a4fbc88a80 PEP 292 classes Template and SafeTemplate are added to the string module. Barry Warsaw <barry@python.org> parents: 31829 diff changeset	255 return LITERAL, int(escape, 8) & 0xff
33397 98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	256 elif c in DIGITS:
43020 907a45df1295 Raise statement normalization in Lib/. Collin Winter <collinw@gmail.com> parents: 41986 diff changeset	257 raise error("bogus escape: %s" % repr(escape))
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	258 if len(escape) == 2:
13201 a093fad92ccf the mad patcher strikes again: Fredrik Lundh <fredrik@pythonware.com> parents: 13198 diff changeset	259 return LITERAL, ord(escape[1])
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	260 except ValueError:
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	261 pass
43020 907a45df1295 Raise statement normalization in Lib/. Collin Winter <collinw@gmail.com> parents: 41986 diff changeset	262 raise error("bogus escape: %s" % repr(escape))
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	263
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	264 def _escape(source, escape, state):
9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	265 # handle escape code in expression
9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	266 code = CATEGORIES.get(escape)
9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	267 if code:
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	268 return code
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	269 code = ESCAPES.get(escape)
9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	270 if code:
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	271 return code
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	272 try:
33397 98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	273 c = escape[1:2]
98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	274 if c == "x":
14316 511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	275 # hexadecimal escape
511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	276 while source.next in HEXDIGITS and len(escape) < 4:
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	277 escape = escape + source.get()
14379 b069d7d9ce8a -- tightened up parsing of octal numbers Fredrik Lundh <fredrik@pythonware.com> parents: 14376 diff changeset	278 if len(escape) != 4:
b069d7d9ce8a -- tightened up parsing of octal numbers Fredrik Lundh <fredrik@pythonware.com> parents: 14376 diff changeset	279 raise ValueError
33257 11a4fbc88a80 PEP 292 classes Template and SafeTemplate are added to the string module. Barry Warsaw <barry@python.org> parents: 31829 diff changeset	280 return LITERAL, int(escape[2:], 16) & 0xff
33397 98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	281 elif c == "0":
14316 511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	282 # octal escape
14379 b069d7d9ce8a -- tightened up parsing of octal numbers Fredrik Lundh <fredrik@pythonware.com> parents: 14376 diff changeset	283 while source.next in OCTDIGITS and len(escape) < 4:
14316 511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	284 escape = escape + source.get()
33257 11a4fbc88a80 PEP 292 classes Template and SafeTemplate are added to the string module. Barry Warsaw <barry@python.org> parents: 31829 diff changeset	285 return LITERAL, int(escape[1:], 8) & 0xff
33397 98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	286 elif c in DIGITS:
14316 511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	287 # octal escape or decimal group reference (sigh)
511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	288 if source.next in DIGITS:
511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	289 escape = escape + source.get()
14379 b069d7d9ce8a -- tightened up parsing of octal numbers Fredrik Lundh <fredrik@pythonware.com> parents: 14376 diff changeset	290 if (escape[1] in OCTDIGITS and escape[2] in OCTDIGITS and
b069d7d9ce8a -- tightened up parsing of octal numbers Fredrik Lundh <fredrik@pythonware.com> parents: 14376 diff changeset	291 source.next in OCTDIGITS):
14316 511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	292 # got three octal digits; this is an octal escape
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	293 escape = escape + source.get()
33257 11a4fbc88a80 PEP 292 classes Template and SafeTemplate are added to the string module. Barry Warsaw <barry@python.org> parents: 31829 diff changeset	294 return LITERAL, int(escape[1:], 8) & 0xff
33397 98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	295 # not an octal escape, so this is a group reference
98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	296 group = int(escape[1:])
98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	297 if group < state.groups:
15355 2a3696d32721 -- properly reset groups in findall (bug #117612) Fredrik Lundh <fredrik@pythonware.com> parents: 15091 diff changeset	298 if not state.checkgroup(group):
43020 907a45df1295 Raise statement normalization in Lib/. Collin Winter <collinw@gmail.com> parents: 41986 diff changeset	299 raise error("cannot refer to open group")
14316 511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	300 return GROUPREF, group
14379 b069d7d9ce8a -- tightened up parsing of octal numbers Fredrik Lundh <fredrik@pythonware.com> parents: 14376 diff changeset	301 raise ValueError
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	302 if len(escape) == 2:
13201 a093fad92ccf the mad patcher strikes again: Fredrik Lundh <fredrik@pythonware.com> parents: 13198 diff changeset	303 return LITERAL, ord(escape[1])
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	304 except ValueError:
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	305 pass
43020 907a45df1295 Raise statement normalization in Lib/. Collin Winter <collinw@gmail.com> parents: 41986 diff changeset	306 raise error("bogus escape: %s" % repr(escape))
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	307
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	308 def _parse_sub(source, state, nested=1):
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	309 # parse an alternation: a\|b\|c
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	310
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	311 items = []
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	312 itemsappend = items.append
a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	313 sourcematch = source.match
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	314 while 1:
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	315 itemsappend(_parse(source, state))
a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	316 if sourcematch("\|"):
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	317 continue
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	318 if not nested:
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	319 break
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	320 if not source.next or sourcematch(")", 0):
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	321 break
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	322 else:
43020 907a45df1295 Raise statement normalization in Lib/. Collin Winter <collinw@gmail.com> parents: 41986 diff changeset	323 raise error("pattern not properly closed")
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	324
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	325 if len(items) == 1:
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	326 return items[0]
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	327
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	328 subpattern = SubPattern(state)
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	329 subpatternappend = subpattern.append
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	330
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	331 # check if all items share a common prefix
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	332 while 1:
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	333 prefix = None
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	334 for item in items:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	335 if not item:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	336 break
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	337 if prefix is None:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	338 prefix = item[0]
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	339 elif item[0] != prefix:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	340 break
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	341 else:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	342 # all subitems start with a common "prefix".
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	343 # move it out of the branch
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	344 for item in items:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	345 del item[0]
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	346 subpatternappend(prefix)
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	347 continue # check next one
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	348 break
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	349
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	350 # check if the branch can be replaced by a character set
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	351 for item in items:
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	352 if len(item) != 1 or item[0][0] != LITERAL:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	353 break
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	354 else:
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	355 # we can store this as a character set instead of a
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	356 # branch (the compiler may optimize this even more)
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	357 set = []
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	358 setappend = set.append
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	359 for item in items:
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	360 setappend(item[0])
a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	361 subpatternappend((IN, set))
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	362 return subpattern
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	363
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	364 subpattern.append((BRANCH, (None, items)))
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	365 return subpattern
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	366
30703 8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	367 def _parse_sub_cond(source, state, condgroup):
31389 aca1ce1b4b61 Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 30703 diff changeset	368 item_yes = _parse(source, state)
30703 8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	369 if source.match("\|"):
31389 aca1ce1b4b61 Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 30703 diff changeset	370 item_no = _parse(source, state)
30703 8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	371 if source.match("\|"):
43020 907a45df1295 Raise statement normalization in Lib/. Collin Winter <collinw@gmail.com> parents: 41986 diff changeset	372 raise error("conditional backref with more than two branches")
30703 8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	373 else:
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	374 item_no = None
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	375 if source.next and not source.match(")", 0):
43020 907a45df1295 Raise statement normalization in Lib/. Collin Winter <collinw@gmail.com> parents: 41986 diff changeset	376 raise error("pattern not properly closed")
30703 8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	377 subpattern = SubPattern(state)
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	378 subpattern.append((GROUPREF_EXISTS, (condgroup, item_yes, item_no)))
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	379 return subpattern
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	380
34635 5f26298811b7 Complete the previous effort to factor out constant expressions Raymond Hettinger <python@rcn.com> parents: 33400 diff changeset	381 _PATTERNENDERS = set("\|)")
5f26298811b7 Complete the previous effort to factor out constant expressions Raymond Hettinger <python@rcn.com> parents: 33400 diff changeset	382 _ASSERTCHARS = set("=!<")
5f26298811b7 Complete the previous effort to factor out constant expressions Raymond Hettinger <python@rcn.com> parents: 33400 diff changeset	383 _LOOKBEHINDASSERTCHARS = set("=!")
5f26298811b7 Complete the previous effort to factor out constant expressions Raymond Hettinger <python@rcn.com> parents: 33400 diff changeset	384 _REPEATCODES = set([MIN_REPEAT, MAX_REPEAT])
5f26298811b7 Complete the previous effort to factor out constant expressions Raymond Hettinger <python@rcn.com> parents: 33400 diff changeset	385
13250 88640ecf5506 - fixed code generation error in multiline mode Fredrik Lundh <fredrik@pythonware.com> parents: 13236 diff changeset	386 def _parse(source, state):
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	387 # parse a simple pattern
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	388 subpattern = SubPattern(state)
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	389
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	390 # precompute constants into local variables
a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	391 subpatternappend = subpattern.append
a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	392 sourceget = source.get
a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	393 sourcematch = source.match
a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	394 _len = len
34635 5f26298811b7 Complete the previous effort to factor out constant expressions Raymond Hettinger <python@rcn.com> parents: 33400 diff changeset	395 PATTERNENDERS = _PATTERNENDERS
5f26298811b7 Complete the previous effort to factor out constant expressions Raymond Hettinger <python@rcn.com> parents: 33400 diff changeset	396 ASSERTCHARS = _ASSERTCHARS
5f26298811b7 Complete the previous effort to factor out constant expressions Raymond Hettinger <python@rcn.com> parents: 33400 diff changeset	397 LOOKBEHINDASSERTCHARS = _LOOKBEHINDASSERTCHARS
5f26298811b7 Complete the previous effort to factor out constant expressions Raymond Hettinger <python@rcn.com> parents: 33400 diff changeset	398 REPEATCODES = _REPEATCODES
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	399
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	400 while 1:
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	401
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	402 if source.next in PATTERNENDERS:
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	403 break # end of subpattern
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	404 this = sourceget()
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	405 if this is None:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	406 break # end of pattern
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	407
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	408 if state.flags & SRE_FLAG_VERBOSE:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	409 # skip whitespace and comments
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	410 if this in WHITESPACE:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	411 continue
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	412 if this == "#":
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	413 while 1:
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	414 this = sourceget()
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	415 if this in (None, "\n"):
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	416 break
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	417 continue
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	418
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	419 if this and this[0] not in SPECIAL_CHARS:
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	420 subpatternappend((LITERAL, ord(this)))
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	421
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	422 elif this == "[":
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	423 # character set
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	424 set = []
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	425 setappend = set.append
a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	426 ## if sourcematch(":"):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	427 ## pass # handle character classes
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	428 if sourcematch("^"):
a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	429 setappend((NEGATE, None))
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	430 # check remaining characters
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	431 start = set[:]
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	432 while 1:
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	433 this = sourceget()
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	434 if this == "]" and set != start:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	435 break
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	436 elif this and this[0] == "\\":
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	437 code1 = _class_escape(source, this)
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	438 elif this:
13201 a093fad92ccf the mad patcher strikes again: Fredrik Lundh <fredrik@pythonware.com> parents: 13198 diff changeset	439 code1 = LITERAL, ord(this)
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	440 else:
43020 907a45df1295 Raise statement normalization in Lib/. Collin Winter <collinw@gmail.com> parents: 41986 diff changeset	441 raise error("unexpected end of regular expression")
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	442 if sourcematch("-"):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	443 # potential range
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	444 this = sourceget()
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	445 if this == "]":
15083 02ad9cb52222 SRE didn't handle character category followed by hyphen inside a Fredrik Lundh <fredrik@pythonware.com> parents: 14999 diff changeset	446 if code1[0] is IN:
02ad9cb52222 SRE didn't handle character category followed by hyphen inside a Fredrik Lundh <fredrik@pythonware.com> parents: 14999 diff changeset	447 code1 = code1[1][0]
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	448 setappend(code1)
a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	449 setappend((LITERAL, ord("-")))
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	450 break
28603 7fe33c1541df SF patch #720991 by Gary Herron: Guido van Rossum <guido@python.org> parents: 23627 diff changeset	451 elif this:
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	452 if this[0] == "\\":
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	453 code2 = _class_escape(source, this)
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	454 else:
13201 a093fad92ccf the mad patcher strikes again: Fredrik Lundh <fredrik@pythonware.com> parents: 13198 diff changeset	455 code2 = LITERAL, ord(this)
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	456 if code1[0] != LITERAL or code2[0] != LITERAL:
43020 907a45df1295 Raise statement normalization in Lib/. Collin Winter <collinw@gmail.com> parents: 41986 diff changeset	457 raise error("bad character range")
14316 511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	458 lo = code1[1]
511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	459 hi = code2[1]
511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	460 if hi < lo:
43020 907a45df1295 Raise statement normalization in Lib/. Collin Winter <collinw@gmail.com> parents: 41986 diff changeset	461 raise error("bad character range")
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	462 setappend((RANGE, (lo, hi)))
28603 7fe33c1541df SF patch #720991 by Gary Herron: Guido van Rossum <guido@python.org> parents: 23627 diff changeset	463 else:
43020 907a45df1295 Raise statement normalization in Lib/. Collin Winter <collinw@gmail.com> parents: 41986 diff changeset	464 raise error("unexpected end of regular expression")
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	465 else:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	466 if code1[0] is IN:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	467 code1 = code1[1][0]
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	468 setappend(code1)
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	469
15855 152d00718006 SRE fixes for 2.1 alpha: Fredrik Lundh <fredrik@pythonware.com> parents: 15355 diff changeset	470 # XXX: <fl> should move set optimization to compiler!
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	471 if _len(set)==1 and set[0][0] is LITERAL:
a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	472 subpatternappend(set[0]) # optimization
a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	473 elif _len(set)==2 and set[0][0] is NEGATE and set[1][0] is LITERAL:
a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	474 subpatternappend((NOT_LITERAL, set[1][1])) # optimization
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	475 else:
15855 152d00718006 SRE fixes for 2.1 alpha: Fredrik Lundh <fredrik@pythonware.com> parents: 15355 diff changeset	476 # XXX: <fl> should add charmap optimization here
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	477 subpatternappend((IN, set))
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	478
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	479 elif this and this[0] in REPEAT_CHARS:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	480 # repeat previous item
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	481 if this == "?":
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	482 min, max = 0, 1
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	483 elif this == "*":
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	484 min, max = 0, MAXREPEAT
16665 de0e8752d5b7 detect attempts to repeat anchors (fixes bug #130748) Fredrik Lundh <fredrik@pythonware.com> parents: 16663 diff changeset	485
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	486 elif this == "+":
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	487 min, max = 1, MAXREPEAT
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	488 elif this == "{":
35544 acfc0e22cd8f Bug #1202493: Fixing SRE parser to handle '{}' as perl does, rather than Gustavo Niemeyer <gustavo@niemeyer.net> parents: 34635 diff changeset	489 if source.next == "}":
acfc0e22cd8f Bug #1202493: Fixing SRE parser to handle '{}' as perl does, rather than Gustavo Niemeyer <gustavo@niemeyer.net> parents: 34635 diff changeset	490 subpatternappend((LITERAL, ord(this)))
acfc0e22cd8f Bug #1202493: Fixing SRE parser to handle '{}' as perl does, rather than Gustavo Niemeyer <gustavo@niemeyer.net> parents: 34635 diff changeset	491 continue
13305 33ce63b06359 - fixed "{ in any other context" bug Fredrik Lundh <fredrik@pythonware.com> parents: 13299 diff changeset	492 here = source.tell()
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	493 min, max = 0, MAXREPEAT
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	494 lo = hi = ""
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	495 while source.next in DIGITS:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	496 lo = lo + source.get()
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	497 if sourcematch(","):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	498 while source.next in DIGITS:
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	499 hi = hi + sourceget()
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	500 else:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	501 hi = lo
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	502 if not sourcematch("}"):
a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	503 subpatternappend((LITERAL, ord(this)))
13305 33ce63b06359 - fixed "{ in any other context" bug Fredrik Lundh <fredrik@pythonware.com> parents: 13299 diff changeset	504 source.seek(here)
33ce63b06359 - fixed "{ in any other context" bug Fredrik Lundh <fredrik@pythonware.com> parents: 13299 diff changeset	505 continue
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	506 if lo:
33257 11a4fbc88a80 PEP 292 classes Template and SafeTemplate are added to the string module. Barry Warsaw <barry@python.org> parents: 31829 diff changeset	507 min = int(lo)
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	508 if hi:
33257 11a4fbc88a80 PEP 292 classes Template and SafeTemplate are added to the string module. Barry Warsaw <barry@python.org> parents: 31829 diff changeset	509 max = int(hi)
15858 81e3d8626adf SRE: stricter pattern syntax checking (covers parts of bug #115900) Fredrik Lundh <fredrik@pythonware.com> parents: 15855 diff changeset	510 if max < min:
43020 907a45df1295 Raise statement normalization in Lib/. Collin Winter <collinw@gmail.com> parents: 41986 diff changeset	511 raise error("bad repeat interval")
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	512 else:
43020 907a45df1295 Raise statement normalization in Lib/. Collin Winter <collinw@gmail.com> parents: 41986 diff changeset	513 raise error("not supported")
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	514 # figure out which item to repeat
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	515 if subpattern:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	516 item = subpattern[-1:]
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	517 else:
16665 de0e8752d5b7 detect attempts to repeat anchors (fixes bug #130748) Fredrik Lundh <fredrik@pythonware.com> parents: 16663 diff changeset	518 item = None
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	519 if not item or (_len(item) == 1 and item[0][0] == AT):
43020 907a45df1295 Raise statement normalization in Lib/. Collin Winter <collinw@gmail.com> parents: 41986 diff changeset	520 raise error("nothing to repeat")
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	521 if item[0][0] in REPEATCODES:
43020 907a45df1295 Raise statement normalization in Lib/. Collin Winter <collinw@gmail.com> parents: 41986 diff changeset	522 raise error("multiple repeat")
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	523 if sourcematch("?"):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	524 subpattern[-1] = (MIN_REPEAT, (min, max, item))
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	525 else:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	526 subpattern[-1] = (MAX_REPEAT, (min, max, item))
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	527
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	528 elif this == ".":
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	529 subpatternappend((ANY, None))
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	530
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	531 elif this == "(":
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	532 group = 1
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	533 name = None
30703 8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	534 condgroup = None
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	535 if sourcematch("?"):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	536 group = 0
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	537 # options
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	538 if sourcematch("P"):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	539 # python extensions
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	540 if sourcematch("<"):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	541 # named group: skip forward to end of name
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	542 name = ""
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	543 while 1:
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	544 char = sourceget()
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	545 if char is None:
43020 907a45df1295 Raise statement normalization in Lib/. Collin Winter <collinw@gmail.com> parents: 41986 diff changeset	546 raise error("unterminated name")
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	547 if char == ">":
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	548 break
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	549 name = name + char
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	550 group = 1
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	551 if not isname(name):
43020 907a45df1295 Raise statement normalization in Lib/. Collin Winter <collinw@gmail.com> parents: 41986 diff changeset	552 raise error("bad character in group name")
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	553 elif sourcematch("="):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	554 # named backreference
13188 92ba928c09b8 - added support for (?P=name) Fredrik Lundh <fredrik@pythonware.com> parents: 13186 diff changeset	555 name = ""
92ba928c09b8 - added support for (?P=name) Fredrik Lundh <fredrik@pythonware.com> parents: 13186 diff changeset	556 while 1:
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	557 char = sourceget()
13188 92ba928c09b8 - added support for (?P=name) Fredrik Lundh <fredrik@pythonware.com> parents: 13186 diff changeset	558 if char is None:
43020 907a45df1295 Raise statement normalization in Lib/. Collin Winter <collinw@gmail.com> parents: 41986 diff changeset	559 raise error("unterminated name")
13188 92ba928c09b8 - added support for (?P=name) Fredrik Lundh <fredrik@pythonware.com> parents: 13186 diff changeset	560 if char == ")":
92ba928c09b8 - added support for (?P=name) Fredrik Lundh <fredrik@pythonware.com> parents: 13186 diff changeset	561 break
92ba928c09b8 - added support for (?P=name) Fredrik Lundh <fredrik@pythonware.com> parents: 13186 diff changeset	562 name = name + char
92ba928c09b8 - added support for (?P=name) Fredrik Lundh <fredrik@pythonware.com> parents: 13186 diff changeset	563 if not isname(name):
43020 907a45df1295 Raise statement normalization in Lib/. Collin Winter <collinw@gmail.com> parents: 41986 diff changeset	564 raise error("bad character in group name")
13188 92ba928c09b8 - added support for (?P=name) Fredrik Lundh <fredrik@pythonware.com> parents: 13186 diff changeset	565 gid = state.groupdict.get(name)
92ba928c09b8 - added support for (?P=name) Fredrik Lundh <fredrik@pythonware.com> parents: 13186 diff changeset	566 if gid is None:
43020 907a45df1295 Raise statement normalization in Lib/. Collin Winter <collinw@gmail.com> parents: 41986 diff changeset	567 raise error("unknown group name")
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	568 subpatternappend((GROUPREF, gid))
13308 fc2ad6fa08ec - actually enabled charset anchors in the engine (still not Fredrik Lundh <fredrik@pythonware.com> parents: 13306 diff changeset	569 continue
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	570 else:
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	571 char = sourceget()
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	572 if char is None:
43020 907a45df1295 Raise statement normalization in Lib/. Collin Winter <collinw@gmail.com> parents: 41986 diff changeset	573 raise error("unexpected end of pattern")
907a45df1295 Raise statement normalization in Lib/. Collin Winter <collinw@gmail.com> parents: 41986 diff changeset	574 raise error("unknown specifier: ?P%s" % char)
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	575 elif sourcematch(":"):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	576 # non-capturing group
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	577 group = 2
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	578 elif sourcematch("#"):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	579 # comment
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	580 while 1:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	581 if source.next is None or source.next == ")":
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	582 break
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	583 sourceget()
a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	584 if not sourcematch(")"):
43020 907a45df1295 Raise statement normalization in Lib/. Collin Winter <collinw@gmail.com> parents: 41986 diff changeset	585 raise error("unbalanced parenthesis")
14316 511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	586 continue
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	587 elif source.next in ASSERTCHARS:
13198 4deeed58bf76 - fixed lookahead assertions (#10, #11, #12) Fredrik Lundh <fredrik@pythonware.com> parents: 13188 diff changeset	588 # lookahead assertions
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	589 char = sourceget()
13327 c155bccb7a22 - added lookbehind support (?<=pattern), (?<!pattern). Fredrik Lundh <fredrik@pythonware.com> parents: 13310 diff changeset	590 dir = 1
c155bccb7a22 - added lookbehind support (?<=pattern), (?<!pattern). Fredrik Lundh <fredrik@pythonware.com> parents: 13310 diff changeset	591 if char == "<":
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	592 if source.next not in LOOKBEHINDASSERTCHARS:
43020 907a45df1295 Raise statement normalization in Lib/. Collin Winter <collinw@gmail.com> parents: 41986 diff changeset	593 raise error("syntax error")
13327 c155bccb7a22 - added lookbehind support (?<=pattern), (?<!pattern). Fredrik Lundh <fredrik@pythonware.com> parents: 13310 diff changeset	594 dir = -1 # lookbehind
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	595 char = sourceget()
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	596 p = _parse_sub(source, state)
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	597 if not sourcematch(")"):
43020 907a45df1295 Raise statement normalization in Lib/. Collin Winter <collinw@gmail.com> parents: 41986 diff changeset	598 raise error("unbalanced parenthesis")
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	599 if char == "=":
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	600 subpatternappend((ASSERT, (dir, p)))
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	601 else:
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	602 subpatternappend((ASSERT_NOT, (dir, p)))
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	603 continue
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	604 elif sourcematch("("):
30703 8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	605 # conditional backreference group
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	606 condname = ""
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	607 while 1:
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	608 char = sourceget()
30703 8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	609 if char is None:
43020 907a45df1295 Raise statement normalization in Lib/. Collin Winter <collinw@gmail.com> parents: 41986 diff changeset	610 raise error("unterminated name")
30703 8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	611 if char == ")":
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	612 break
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	613 condname = condname + char
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	614 group = 2
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	615 if isname(condname):
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	616 condgroup = state.groupdict.get(condname)
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	617 if condgroup is None:
43020 907a45df1295 Raise statement normalization in Lib/. Collin Winter <collinw@gmail.com> parents: 41986 diff changeset	618 raise error("unknown group name")
30703 8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	619 else:
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	620 try:
33257 11a4fbc88a80 PEP 292 classes Template and SafeTemplate are added to the string module. Barry Warsaw <barry@python.org> parents: 31829 diff changeset	621 condgroup = int(condname)
30703 8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	622 except ValueError:
43020 907a45df1295 Raise statement normalization in Lib/. Collin Winter <collinw@gmail.com> parents: 41986 diff changeset	623 raise error("bad character in group name")
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	624 else:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	625 # flags
23620 a1de0bab5475 SF 563203. Replaced 'has_key()' with 'in'. Raymond Hettinger <python@rcn.com> parents: 22867 diff changeset	626 if not source.next in FLAGS:
43020 907a45df1295 Raise statement normalization in Lib/. Collin Winter <collinw@gmail.com> parents: 41986 diff changeset	627 raise error("unexpected end of pattern")
23620 a1de0bab5475 SF 563203. Replaced 'has_key()' with 'in'. Raymond Hettinger <python@rcn.com> parents: 22867 diff changeset	628 while source.next in FLAGS:
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	629 state.flags = state.flags \| FLAGS[sourceget()]
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	630 if group:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	631 # parse group contents
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	632 if group == 2:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	633 # anonymous group
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	634 group = None
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	635 else:
15355 2a3696d32721 -- properly reset groups in findall (bug #117612) Fredrik Lundh <fredrik@pythonware.com> parents: 15091 diff changeset	636 group = state.opengroup(name)
30703 8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	637 if condgroup:
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	638 p = _parse_sub_cond(source, state, condgroup)
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	639 else:
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	640 p = _parse_sub(source, state)
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	641 if not sourcematch(")"):
43020 907a45df1295 Raise statement normalization in Lib/. Collin Winter <collinw@gmail.com> parents: 41986 diff changeset	642 raise error("unbalanced parenthesis")
15355 2a3696d32721 -- properly reset groups in findall (bug #117612) Fredrik Lundh <fredrik@pythonware.com> parents: 15091 diff changeset	643 if group is not None:
2a3696d32721 -- properly reset groups in findall (bug #117612) Fredrik Lundh <fredrik@pythonware.com> parents: 15091 diff changeset	644 state.closegroup(group)
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	645 subpatternappend((SUBPATTERN, (group, p)))
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	646 else:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	647 while 1:
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	648 char = sourceget()
15858 81e3d8626adf SRE: stricter pattern syntax checking (covers parts of bug #115900) Fredrik Lundh <fredrik@pythonware.com> parents: 15855 diff changeset	649 if char is None:
43020 907a45df1295 Raise statement normalization in Lib/. Collin Winter <collinw@gmail.com> parents: 41986 diff changeset	650 raise error("unexpected end of pattern")
15858 81e3d8626adf SRE: stricter pattern syntax checking (covers parts of bug #115900) Fredrik Lundh <fredrik@pythonware.com> parents: 15855 diff changeset	651 if char == ")":
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	652 break
43020 907a45df1295 Raise statement normalization in Lib/. Collin Winter <collinw@gmail.com> parents: 41986 diff changeset	653 raise error("unknown extension")
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	654
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	655 elif this == "^":
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	656 subpatternappend((AT, AT_BEGINNING))
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	657
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	658 elif this == "$":
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	659 subpattern.append((AT, AT_END))
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	660
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	661 elif this and this[0] == "\\":
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	662 code = _escape(source, this, state)
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	663 subpatternappend(code)
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	664
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	665 else:
43020 907a45df1295 Raise statement normalization in Lib/. Collin Winter <collinw@gmail.com> parents: 41986 diff changeset	666 raise error("parser error")
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	667
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	668 return subpattern
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	669
48769 83b2d07b475b #2834: Change re module semantics, so that str and bytes mixing is forbidden, Antoine Pitrou <solipsis@pitrou.net> parents: 48554 diff changeset	670 def fix_flags(src, flags):
83b2d07b475b #2834: Change re module semantics, so that str and bytes mixing is forbidden, Antoine Pitrou <solipsis@pitrou.net> parents: 48554 diff changeset	671 # Check and fix flags according to the type of pattern (str or bytes)
83b2d07b475b #2834: Change re module semantics, so that str and bytes mixing is forbidden, Antoine Pitrou <solipsis@pitrou.net> parents: 48554 diff changeset	672 if isinstance(src, str):
83b2d07b475b #2834: Change re module semantics, so that str and bytes mixing is forbidden, Antoine Pitrou <solipsis@pitrou.net> parents: 48554 diff changeset	673 if not flags & SRE_FLAG_ASCII:
83b2d07b475b #2834: Change re module semantics, so that str and bytes mixing is forbidden, Antoine Pitrou <solipsis@pitrou.net> parents: 48554 diff changeset	674 flags \|= SRE_FLAG_UNICODE
83b2d07b475b #2834: Change re module semantics, so that str and bytes mixing is forbidden, Antoine Pitrou <solipsis@pitrou.net> parents: 48554 diff changeset	675 elif flags & SRE_FLAG_UNICODE:
83b2d07b475b #2834: Change re module semantics, so that str and bytes mixing is forbidden, Antoine Pitrou <solipsis@pitrou.net> parents: 48554 diff changeset	676 raise ValueError("ASCII and UNICODE flags are incompatible")
83b2d07b475b #2834: Change re module semantics, so that str and bytes mixing is forbidden, Antoine Pitrou <solipsis@pitrou.net> parents: 48554 diff changeset	677 else:
83b2d07b475b #2834: Change re module semantics, so that str and bytes mixing is forbidden, Antoine Pitrou <solipsis@pitrou.net> parents: 48554 diff changeset	678 if flags & SRE_FLAG_UNICODE:
83b2d07b475b #2834: Change re module semantics, so that str and bytes mixing is forbidden, Antoine Pitrou <solipsis@pitrou.net> parents: 48554 diff changeset	679 raise ValueError("can't use UNICODE flag with a bytes pattern")
83b2d07b475b #2834: Change re module semantics, so that str and bytes mixing is forbidden, Antoine Pitrou <solipsis@pitrou.net> parents: 48554 diff changeset	680 return flags
83b2d07b475b #2834: Change re module semantics, so that str and bytes mixing is forbidden, Antoine Pitrou <solipsis@pitrou.net> parents: 48554 diff changeset	681
13886 62494a96dcca -- reset marks if repeat_one tail doesn't match Fredrik Lundh <fredrik@pythonware.com> parents: 13795 diff changeset	682 def parse(str, flags=0, pattern=None):
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	683 # parse 're' pattern into list of (opcode, argument) tuples
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	684
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	685 source = Tokenizer(str)
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	686
13886 62494a96dcca -- reset marks if repeat_one tail doesn't match Fredrik Lundh <fredrik@pythonware.com> parents: 13795 diff changeset	687 if pattern is None:
62494a96dcca -- reset marks if repeat_one tail doesn't match Fredrik Lundh <fredrik@pythonware.com> parents: 13795 diff changeset	688 pattern = Pattern()
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	689 pattern.flags = flags
15858 81e3d8626adf SRE: stricter pattern syntax checking (covers parts of bug #115900) Fredrik Lundh <fredrik@pythonware.com> parents: 15855 diff changeset	690 pattern.str = str
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	691
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	692 p = _parse_sub(source, pattern, 0)
48769 83b2d07b475b #2834: Change re module semantics, so that str and bytes mixing is forbidden, Antoine Pitrou <solipsis@pitrou.net> parents: 48554 diff changeset	693 p.pattern.flags = fix_flags(str, p.pattern.flags)
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	694
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	695 tail = source.get()
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	696 if tail == ")":
43020 907a45df1295 Raise statement normalization in Lib/. Collin Winter <collinw@gmail.com> parents: 41986 diff changeset	697 raise error("unbalanced parenthesis")
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	698 elif tail:
43020 907a45df1295 Raise statement normalization in Lib/. Collin Winter <collinw@gmail.com> parents: 41986 diff changeset	699 raise error("bogus characters at end of regular expression")
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	700
15855 152d00718006 SRE fixes for 2.1 alpha: Fredrik Lundh <fredrik@pythonware.com> parents: 15355 diff changeset	701 if flags & SRE_FLAG_DEBUG:
152d00718006 SRE fixes for 2.1 alpha: Fredrik Lundh <fredrik@pythonware.com> parents: 15355 diff changeset	702 p.dump()
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	703
14999 1a3b5c8d6730 Recompile pattern if (?x) flag was found inside the pattern during the Fredrik Lundh <fredrik@pythonware.com> parents: 14797 diff changeset	704 if not (flags & SRE_FLAG_VERBOSE) and p.pattern.flags & SRE_FLAG_VERBOSE:
1a3b5c8d6730 Recompile pattern if (?x) flag was found inside the pattern during the Fredrik Lundh <fredrik@pythonware.com> parents: 14797 diff changeset	705 # the VERBOSE flag was switched on inside the pattern. to be
1a3b5c8d6730 Recompile pattern if (?x) flag was found inside the pattern during the Fredrik Lundh <fredrik@pythonware.com> parents: 14797 diff changeset	706 # on the safe side, we'll parse the whole thing again...
1a3b5c8d6730 Recompile pattern if (?x) flag was found inside the pattern during the Fredrik Lundh <fredrik@pythonware.com> parents: 14797 diff changeset	707 return parse(str, p.pattern.flags)
1a3b5c8d6730 Recompile pattern if (?x) flag was found inside the pattern during the Fredrik Lundh <fredrik@pythonware.com> parents: 14797 diff changeset	708
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	709 return p
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	710
13118 7ff40da08c2a towards 1.6b1 Fredrik Lundh <fredrik@pythonware.com> parents: 12903 diff changeset	711 def parse_template(source, pattern):
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	712 # parse 're' replacement string into list of literals and
9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	713 # group references
9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	714 s = Tokenizer(source)
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	715 sget = s.get
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	716 p = []
9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	717 a = p.append
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	718 def literal(literal, p=p, pappend=a):
17144 bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	719 if p and p[-1][0] is LITERAL:
bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	720 p[-1] = LITERAL, p[-1][1] + literal
bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	721 else:
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	722 pappend((LITERAL, literal))
17144 bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	723 sep = source[:0]
41986 6464cc638faa Merged revisions 55795-55816 via svnmerge from Guido van Rossum <guido@python.org> parents: 41585 diff changeset	724 if isinstance(sep, str):
19971 d4c965316610 fixed #449964: sre.sub raises an exception if the template contains a Fredrik Lundh <fredrik@pythonware.com> parents: 19693 diff changeset	725 makechar = chr
17144 bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	726 else:
41585 825d966338ea Rename 'unicode' to 'str' in its tp_name field. Rename 'str' to 'str8'. Guido van Rossum <guido@python.org> parents: 40775 diff changeset	727 makechar = chr
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	728 while 1:
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	729 this = sget()
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	730 if this is None:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	731 break # end of replacement string
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	732 if this and this[0] == "\\":
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	733 # group
33397 98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	734 c = this[1:2]
98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	735 if c == "g":
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	736 name = ""
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	737 if s.match("<"):
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	738 while 1:
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	739 char = sget()
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	740 if char is None:
43020 907a45df1295 Raise statement normalization in Lib/. Collin Winter <collinw@gmail.com> parents: 41986 diff changeset	741 raise error("unterminated group name")
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	742 if char == ">":
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	743 break
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	744 name = name + char
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	745 if not name:
43020 907a45df1295 Raise statement normalization in Lib/. Collin Winter <collinw@gmail.com> parents: 41986 diff changeset	746 raise error("bad group name")
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	747 try:
33257 11a4fbc88a80 PEP 292 classes Template and SafeTemplate are added to the string module. Barry Warsaw <barry@python.org> parents: 31829 diff changeset	748 index = int(name)
33397 98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	749 if index < 0:
43020 907a45df1295 Raise statement normalization in Lib/. Collin Winter <collinw@gmail.com> parents: 41986 diff changeset	750 raise error("negative group number")
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	751 except ValueError:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	752 if not isname(name):
43020 907a45df1295 Raise statement normalization in Lib/. Collin Winter <collinw@gmail.com> parents: 41986 diff changeset	753 raise error("bad character in group name")
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	754 try:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	755 index = pattern.groupindex[name]
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	756 except KeyError:
43020 907a45df1295 Raise statement normalization in Lib/. Collin Winter <collinw@gmail.com> parents: 41986 diff changeset	757 raise IndexError("unknown group name")
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	758 a((MARK, index))
33397 98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	759 elif c == "0":
98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	760 if s.next in OCTDIGITS:
98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	761 this = this + sget()
98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	762 if s.next in OCTDIGITS:
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	763 this = this + sget()
33397 98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	764 literal(makechar(int(this[1:], 8) & 0xff))
98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	765 elif c in DIGITS:
98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	766 isoctal = False
98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	767 if s.next in DIGITS:
98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	768 this = this + sget()
33400 b9945335da27 Minor aesthetic change in sre_parse.py, as suggested by Mike Coleman. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33397 diff changeset	769 if (c in OCTDIGITS and this[2] in OCTDIGITS and
b9945335da27 Minor aesthetic change in sre_parse.py, as suggested by Mike Coleman. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33397 diff changeset	770 s.next in OCTDIGITS):
33397 98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	771 this = this + sget()
98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	772 isoctal = True
98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	773 literal(makechar(int(this[1:], 8) & 0xff))
98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	774 if not isoctal:
98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	775 a((MARK, int(this[1:])))
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	776 else:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	777 try:
19971 d4c965316610 fixed #449964: sre.sub raises an exception if the template contains a Fredrik Lundh <fredrik@pythonware.com> parents: 19693 diff changeset	778 this = makechar(ESCAPES[this][1])
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	779 except KeyError:
17144 bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	780 pass
bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	781 literal(this)
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	782 else:
17144 bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	783 literal(this)
bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	784 # convert template to groups and literals lists
bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	785 i = 0
bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	786 groups = []
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	787 groupsappend = groups.append
a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	788 literals = [None] * len(p)
59403 9258f405a7ac #6509: fix re.sub to work properly when the pattern, the string, and the replacement were all bytes. Patch by Antoine Pitrou. Ezio Melotti <ezio.melotti@gmail.com> parents: 49504 diff changeset	789 if isinstance(source, str):
9258f405a7ac #6509: fix re.sub to work properly when the pattern, the string, and the replacement were all bytes. Patch by Antoine Pitrou. Ezio Melotti <ezio.melotti@gmail.com> parents: 49504 diff changeset	790 encode = lambda x: x
9258f405a7ac #6509: fix re.sub to work properly when the pattern, the string, and the replacement were all bytes. Patch by Antoine Pitrou. Ezio Melotti <ezio.melotti@gmail.com> parents: 49504 diff changeset	791 else:
9258f405a7ac #6509: fix re.sub to work properly when the pattern, the string, and the replacement were all bytes. Patch by Antoine Pitrou. Ezio Melotti <ezio.melotti@gmail.com> parents: 49504 diff changeset	792 # The tokenizer implicitly decodes bytes objects as latin-1, we must
9258f405a7ac #6509: fix re.sub to work properly when the pattern, the string, and the replacement were all bytes. Patch by Antoine Pitrou. Ezio Melotti <ezio.melotti@gmail.com> parents: 49504 diff changeset	793 # therefore re-encode the final representation.
9258f405a7ac #6509: fix re.sub to work properly when the pattern, the string, and the replacement were all bytes. Patch by Antoine Pitrou. Ezio Melotti <ezio.melotti@gmail.com> parents: 49504 diff changeset	794 encode = lambda x: x.encode('latin1')
17144 bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	795 for c, s in p:
bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	796 if c is MARK:
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	797 groupsappend((i, s))
a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	798 # literal[i] is already None
17144 bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	799 else:
59403 9258f405a7ac #6509: fix re.sub to work properly when the pattern, the string, and the replacement were all bytes. Patch by Antoine Pitrou. Ezio Melotti <ezio.melotti@gmail.com> parents: 49504 diff changeset	800 literals[i] = encode(s)
17144 bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	801 i = i + 1
bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	802 return groups, literals
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	803
13118 7ff40da08c2a towards 1.6b1 Fredrik Lundh <fredrik@pythonware.com> parents: 12903 diff changeset	804 def expand_template(template, match):
17144 bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	805 g = match.group
13201 a093fad92ccf the mad patcher strikes again: Fredrik Lundh <fredrik@pythonware.com> parents: 13198 diff changeset	806 sep = match.string[:0]
17144 bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	807 groups, literals = template
bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	808 literals = literals[:]
bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	809 try:
bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	810 for index, group in groups:
bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	811 literals[index] = s = g(group)
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	812 if s is None:
43020 907a45df1295 Raise statement normalization in Lib/. Collin Winter <collinw@gmail.com> parents: 41986 diff changeset	813 raise error("unmatched group")
17144 bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	814 except IndexError:
43020 907a45df1295 Raise statement normalization in Lib/. Collin Winter <collinw@gmail.com> parents: 41986 diff changeset	815 raise error("invalid group reference")
33257 11a4fbc88a80 PEP 292 classes Template and SafeTemplate are added to the string module. Barry Warsaw <barry@python.org> parents: 31829 diff changeset	816 return sep.join(literals)

Mercurial > cpython-lru_cache-weakref

annotate Lib/sre_parse.py @ 78069:a8aa77458e12 3.2