cpython-lru_cache-weakref: Lib/sre

annotate Lib/sre_parse.py @ 79675:c0a2294de787 2.7

Issue #16199: Backport 2to3 typo fix from 3.2.

author	Chris Jerdonek <chris.jerdonek@gmail.com>
date	Thu, 11 Oct 2012 16:15:46 -0700
parents	b09e7053b472
children	5b5c2c3b2268

rev	line source
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	1 #
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	2 # Secret Labs' Regular Expression Engine
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	3 #
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	4 # convert re-style regular expression to sre pattern
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	5 #
15855 152d00718006 SRE fixes for 2.1 alpha: Fredrik Lundh <fredrik@pythonware.com> parents: 15355 diff changeset	6 # Copyright (c) 1998-2001 by Secret Labs AB. All rights reserved.
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	7 #
13789 4edd13eb759d SRE 0.9.8: passes the entire test suite Fredrik Lundh <fredrik@pythonware.com> parents: 13677 diff changeset	8 # See the sre.py file for information on usage and redistribution.
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	9 #
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	10
19693 617f5e72b0ec Added docstrings by Neal Norwitz. This closes SF bug #450980. Fred Drake <fdrake@acm.org> parents: 17144 diff changeset	11 """Internal support module for sre"""
617f5e72b0ec Added docstrings by Neal Norwitz. This closes SF bug #450980. Fred Drake <fdrake@acm.org> parents: 17144 diff changeset	12
15858 81e3d8626adf SRE: stricter pattern syntax checking (covers parts of bug #115900) Fredrik Lundh <fredrik@pythonware.com> parents: 15855 diff changeset	13 # XXX: show string offset and offending character for all errors
81e3d8626adf SRE: stricter pattern syntax checking (covers parts of bug #115900) Fredrik Lundh <fredrik@pythonware.com> parents: 15855 diff changeset	14
33257 11a4fbc88a80 PEP 292 classes Template and SafeTemplate are added to the string module. Barry Warsaw <barry@python.org> parents: 31829 diff changeset	15 import sys
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	16
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	17 from sre_constants import *
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	18
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	19 SPECIAL_CHARS = ".\\[{()*+?^$\|"
14379 b069d7d9ce8a -- tightened up parsing of octal numbers Fredrik Lundh <fredrik@pythonware.com> parents: 14376 diff changeset	20 REPEAT_CHARS = "*+?{"
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	21
34635 5f26298811b7 Complete the previous effort to factor out constant expressions Raymond Hettinger <python@rcn.com> parents: 33400 diff changeset	22 DIGITS = set("0123456789")
12339 9e9666f679f7 Fredrik Lundh: new snapshot. Mostly reindented. Guido van Rossum <guido@python.org> parents: 12213 diff changeset	23
34635 5f26298811b7 Complete the previous effort to factor out constant expressions Raymond Hettinger <python@rcn.com> parents: 33400 diff changeset	24 OCTDIGITS = set("01234567")
5f26298811b7 Complete the previous effort to factor out constant expressions Raymond Hettinger <python@rcn.com> parents: 33400 diff changeset	25 HEXDIGITS = set("0123456789abcdefABCDEF")
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	26
34635 5f26298811b7 Complete the previous effort to factor out constant expressions Raymond Hettinger <python@rcn.com> parents: 33400 diff changeset	27 WHITESPACE = set(" \t\n\r\v\f")
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	28
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	29 ESCAPES = {
16663 94a38443f99b - restored 1.5.2 compatibility (sorry, eric) Fredrik Lundh <fredrik@pythonware.com> parents: 16613 diff changeset	30 r"\a": (LITERAL, ord("\a")),
94a38443f99b - restored 1.5.2 compatibility (sorry, eric) Fredrik Lundh <fredrik@pythonware.com> parents: 16613 diff changeset	31 r"\b": (LITERAL, ord("\b")),
94a38443f99b - restored 1.5.2 compatibility (sorry, eric) Fredrik Lundh <fredrik@pythonware.com> parents: 16613 diff changeset	32 r"\f": (LITERAL, ord("\f")),
94a38443f99b - restored 1.5.2 compatibility (sorry, eric) Fredrik Lundh <fredrik@pythonware.com> parents: 16613 diff changeset	33 r"\n": (LITERAL, ord("\n")),
94a38443f99b - restored 1.5.2 compatibility (sorry, eric) Fredrik Lundh <fredrik@pythonware.com> parents: 16613 diff changeset	34 r"\r": (LITERAL, ord("\r")),
94a38443f99b - restored 1.5.2 compatibility (sorry, eric) Fredrik Lundh <fredrik@pythonware.com> parents: 16613 diff changeset	35 r"\t": (LITERAL, ord("\t")),
94a38443f99b - restored 1.5.2 compatibility (sorry, eric) Fredrik Lundh <fredrik@pythonware.com> parents: 16613 diff changeset	36 r"\v": (LITERAL, ord("\v")),
13201 a093fad92ccf the mad patcher strikes again: Fredrik Lundh <fredrik@pythonware.com> parents: 13198 diff changeset	37 r"\\": (LITERAL, ord("\\"))
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	38 }
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	39
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	40 CATEGORIES = {
15855 152d00718006 SRE fixes for 2.1 alpha: Fredrik Lundh <fredrik@pythonware.com> parents: 15355 diff changeset	41 r"\A": (AT, AT_BEGINNING_STRING), # start of string
13166 023a2018b96d - fixed split behaviour on empty matches Fredrik Lundh <fredrik@pythonware.com> parents: 13139 diff changeset	42 r"\b": (AT, AT_BOUNDARY),
023a2018b96d - fixed split behaviour on empty matches Fredrik Lundh <fredrik@pythonware.com> parents: 13139 diff changeset	43 r"\B": (AT, AT_NON_BOUNDARY),
023a2018b96d - fixed split behaviour on empty matches Fredrik Lundh <fredrik@pythonware.com> parents: 13139 diff changeset	44 r"\d": (IN, [(CATEGORY, CATEGORY_DIGIT)]),
023a2018b96d - fixed split behaviour on empty matches Fredrik Lundh <fredrik@pythonware.com> parents: 13139 diff changeset	45 r"\D": (IN, [(CATEGORY, CATEGORY_NOT_DIGIT)]),
023a2018b96d - fixed split behaviour on empty matches Fredrik Lundh <fredrik@pythonware.com> parents: 13139 diff changeset	46 r"\s": (IN, [(CATEGORY, CATEGORY_SPACE)]),
023a2018b96d - fixed split behaviour on empty matches Fredrik Lundh <fredrik@pythonware.com> parents: 13139 diff changeset	47 r"\S": (IN, [(CATEGORY, CATEGORY_NOT_SPACE)]),
023a2018b96d - fixed split behaviour on empty matches Fredrik Lundh <fredrik@pythonware.com> parents: 13139 diff changeset	48 r"\w": (IN, [(CATEGORY, CATEGORY_WORD)]),
023a2018b96d - fixed split behaviour on empty matches Fredrik Lundh <fredrik@pythonware.com> parents: 13139 diff changeset	49 r"\W": (IN, [(CATEGORY, CATEGORY_NOT_WORD)]),
15855 152d00718006 SRE fixes for 2.1 alpha: Fredrik Lundh <fredrik@pythonware.com> parents: 15355 diff changeset	50 r"\Z": (AT, AT_END_STRING), # end of string
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	51 }
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	52
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	53 FLAGS = {
13118 7ff40da08c2a towards 1.6b1 Fredrik Lundh <fredrik@pythonware.com> parents: 12903 diff changeset	54 # standard flags
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	55 "i": SRE_FLAG_IGNORECASE,
9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	56 "L": SRE_FLAG_LOCALE,
9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	57 "m": SRE_FLAG_MULTILINE,
9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	58 "s": SRE_FLAG_DOTALL,
13118 7ff40da08c2a towards 1.6b1 Fredrik Lundh <fredrik@pythonware.com> parents: 12903 diff changeset	59 "x": SRE_FLAG_VERBOSE,
7ff40da08c2a towards 1.6b1 Fredrik Lundh <fredrik@pythonware.com> parents: 12903 diff changeset	60 # extensions
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	61 "t": SRE_FLAG_TEMPLATE,
13118 7ff40da08c2a towards 1.6b1 Fredrik Lundh <fredrik@pythonware.com> parents: 12903 diff changeset	62 "u": SRE_FLAG_UNICODE,
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	63 }
9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	64
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	65 class Pattern:
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	66 # master pattern object. keeps track of global attributes
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	67 def __init__(self):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	68 self.flags = 0
15355 2a3696d32721 -- properly reset groups in findall (bug #117612) Fredrik Lundh <fredrik@pythonware.com> parents: 15091 diff changeset	69 self.open = []
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	70 self.groups = 1
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	71 self.groupdict = {}
15355 2a3696d32721 -- properly reset groups in findall (bug #117612) Fredrik Lundh <fredrik@pythonware.com> parents: 15091 diff changeset	72 def opengroup(self, name=None):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	73 gid = self.groups
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	74 self.groups = gid + 1
23627 a6a765c7d33b Replace boolean test with is None. Raymond Hettinger <python@rcn.com> parents: 23620 diff changeset	75 if name is not None:
20860 0caea5fefa5c Improved error msg when a symbolic group name is redefined. Added docs Tim Peters <tim.peters@gmail.com> parents: 20830 diff changeset	76 ogid = self.groupdict.get(name, None)
0caea5fefa5c Improved error msg when a symbolic group name is redefined. Added docs Tim Peters <tim.peters@gmail.com> parents: 20830 diff changeset	77 if ogid is not None:
21319 4bcfb9dc57e0 bug #133283, #477728, #483789, #490573 Fredrik Lundh <fredrik@pythonware.com> parents: 20860 diff changeset	78 raise error, ("redefinition of group name %s as group %d; "
4bcfb9dc57e0 bug #133283, #477728, #483789, #490573 Fredrik Lundh <fredrik@pythonware.com> parents: 20860 diff changeset	79 "was group %d" % (repr(name), gid, ogid))
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	80 self.groupdict[name] = gid
15355 2a3696d32721 -- properly reset groups in findall (bug #117612) Fredrik Lundh <fredrik@pythonware.com> parents: 15091 diff changeset	81 self.open.append(gid)
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	82 return gid
15355 2a3696d32721 -- properly reset groups in findall (bug #117612) Fredrik Lundh <fredrik@pythonware.com> parents: 15091 diff changeset	83 def closegroup(self, gid):
2a3696d32721 -- properly reset groups in findall (bug #117612) Fredrik Lundh <fredrik@pythonware.com> parents: 15091 diff changeset	84 self.open.remove(gid)
2a3696d32721 -- properly reset groups in findall (bug #117612) Fredrik Lundh <fredrik@pythonware.com> parents: 15091 diff changeset	85 def checkgroup(self, gid):
2a3696d32721 -- properly reset groups in findall (bug #117612) Fredrik Lundh <fredrik@pythonware.com> parents: 15091 diff changeset	86 return gid < self.groups and gid not in self.open
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	87
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	88 class SubPattern:
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	89 # a subpattern, in intermediate form
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	90 def __init__(self, pattern, data=None):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	91 self.pattern = pattern
23627 a6a765c7d33b Replace boolean test with is None. Raymond Hettinger <python@rcn.com> parents: 23620 diff changeset	92 if data is None:
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	93 data = []
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	94 self.data = data
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	95 self.width = None
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	96 def dump(self, level=0):
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	97 nl = 1
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	98 seqtypes = type(()), type([])
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	99 for op, av in self.data:
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	100 print level*" " + op,; nl = 0
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	101 if op == "in":
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	102 # member sublanguage
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	103 print; nl = 1
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	104 for op, a in av:
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	105 print (level+1)*" " + op, a
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	106 elif op == "branch":
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	107 print; nl = 1
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	108 i = 0
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	109 for a in av[1]:
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	110 if i > 0:
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	111 print level*" " + "or"
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	112 a.dump(level+1); nl = 1
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	113 i = i + 1
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	114 elif type(av) in seqtypes:
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	115 for a in av:
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	116 if isinstance(a, SubPattern):
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	117 if not nl: print
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	118 a.dump(level+1); nl = 1
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	119 else:
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	120 print a, ; nl = 0
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	121 else:
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	122 print av, ; nl = 0
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	123 if not nl: print
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	124 def __repr__(self):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	125 return repr(self.data)
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	126 def __len__(self):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	127 return len(self.data)
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	128 def __delitem__(self, index):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	129 del self.data[index]
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	130 def __getitem__(self, index):
40471 eb3b239a4d07 Make sre's SubPattern objects accept slice objects like it already accepts Thomas Wouters <thomas@python.org> parents: 35544 diff changeset	131 if isinstance(index, slice):
eb3b239a4d07 Make sre's SubPattern objects accept slice objects like it already accepts Thomas Wouters <thomas@python.org> parents: 35544 diff changeset	132 return SubPattern(self.pattern, self.data[index])
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	133 return self.data[index]
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	134 def __setitem__(self, index, code):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	135 self.data[index] = code
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	136 def insert(self, index, code):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	137 self.data.insert(index, code)
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	138 def append(self, code):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	139 self.data.append(code)
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	140 def getwidth(self):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	141 # determine the width (min, max) for this subpattern
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	142 if self.width:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	143 return self.width
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	144 lo = hi = 0L
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	145 UNITCODES = (ANY, RANGE, IN, LITERAL, NOT_LITERAL, CATEGORY)
a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	146 REPEATCODES = (MIN_REPEAT, MAX_REPEAT)
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	147 for op, av in self.data:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	148 if op is BRANCH:
13793 4c10bc811fed -- fixed width calculations for alternations Fredrik Lundh <fredrik@pythonware.com> parents: 13789 diff changeset	149 i = sys.maxint
4c10bc811fed -- fixed width calculations for alternations Fredrik Lundh <fredrik@pythonware.com> parents: 13789 diff changeset	150 j = 0
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	151 for av in av[1]:
13793 4c10bc811fed -- fixed width calculations for alternations Fredrik Lundh <fredrik@pythonware.com> parents: 13789 diff changeset	152 l, h = av.getwidth()
4c10bc811fed -- fixed width calculations for alternations Fredrik Lundh <fredrik@pythonware.com> parents: 13789 diff changeset	153 i = min(i, l)
13795 8295f8506eae final 0.9.8 updates: Fredrik Lundh <fredrik@pythonware.com> parents: 13793 diff changeset	154 j = max(j, h)
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	155 lo = lo + i
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	156 hi = hi + j
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	157 elif op is CALL:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	158 i, j = av.getwidth()
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	159 lo = lo + i
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	160 hi = hi + j
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	161 elif op is SUBPATTERN:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	162 i, j = av[1].getwidth()
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	163 lo = lo + i
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	164 hi = hi + j
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	165 elif op in REPEATCODES:
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	166 i, j = av[2].getwidth()
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	167 lo = lo + long(i) * av[0]
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	168 hi = hi + long(j) * av[1]
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	169 elif op in UNITCODES:
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	170 lo = lo + 1
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	171 hi = hi + 1
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	172 elif op == SUCCESS:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	173 break
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	174 self.width = int(min(lo, sys.maxint)), int(min(hi, sys.maxint))
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	175 return self.width
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	176
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	177 class Tokenizer:
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	178 def __init__(self, string):
13305 33ce63b06359 - fixed "{ in any other context" bug Fredrik Lundh <fredrik@pythonware.com> parents: 13299 diff changeset	179 self.string = string
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	180 self.index = 0
13305 33ce63b06359 - fixed "{ in any other context" bug Fredrik Lundh <fredrik@pythonware.com> parents: 13299 diff changeset	181 self.__next()
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	182 def __next(self):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	183 if self.index >= len(self.string):
13305 33ce63b06359 - fixed "{ in any other context" bug Fredrik Lundh <fredrik@pythonware.com> parents: 13299 diff changeset	184 self.next = None
33ce63b06359 - fixed "{ in any other context" bug Fredrik Lundh <fredrik@pythonware.com> parents: 13299 diff changeset	185 return
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	186 char = self.string[self.index]
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	187 if char[0] == "\\":
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	188 try:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	189 c = self.string[self.index + 1]
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	190 except IndexError:
20830 5f24b20e103d SF bug #476912: flag repeated use of the same groupname as Fredrik Lundh <fredrik@pythonware.com> parents: 19971 diff changeset	191 raise error, "bogus escape (end of line)"
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	192 char = char + c
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	193 self.index = self.index + len(char)
13305 33ce63b06359 - fixed "{ in any other context" bug Fredrik Lundh <fredrik@pythonware.com> parents: 13299 diff changeset	194 self.next = char
14316 511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	195 def match(self, char, skip=1):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	196 if char == self.next:
14316 511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	197 if skip:
511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	198 self.__next()
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	199 return 1
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	200 return 0
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	201 def get(self):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	202 this = self.next
13305 33ce63b06359 - fixed "{ in any other context" bug Fredrik Lundh <fredrik@pythonware.com> parents: 13299 diff changeset	203 self.__next()
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	204 return this
13305 33ce63b06359 - fixed "{ in any other context" bug Fredrik Lundh <fredrik@pythonware.com> parents: 13299 diff changeset	205 def tell(self):
33ce63b06359 - fixed "{ in any other context" bug Fredrik Lundh <fredrik@pythonware.com> parents: 13299 diff changeset	206 return self.index, self.next
33ce63b06359 - fixed "{ in any other context" bug Fredrik Lundh <fredrik@pythonware.com> parents: 13299 diff changeset	207 def seek(self, index):
33ce63b06359 - fixed "{ in any other context" bug Fredrik Lundh <fredrik@pythonware.com> parents: 13299 diff changeset	208 self.index, self.next = index
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	209
13122 53d00e7c5848 - make sure group names are valid identifiers Fredrik Lundh <fredrik@pythonware.com> parents: 13121 diff changeset	210 def isident(char):
53d00e7c5848 - make sure group names are valid identifiers Fredrik Lundh <fredrik@pythonware.com> parents: 13121 diff changeset	211 return "a" <= char <= "z" or "A" <= char <= "Z" or char == "_"
53d00e7c5848 - make sure group names are valid identifiers Fredrik Lundh <fredrik@pythonware.com> parents: 13121 diff changeset	212
53d00e7c5848 - make sure group names are valid identifiers Fredrik Lundh <fredrik@pythonware.com> parents: 13121 diff changeset	213 def isdigit(char):
53d00e7c5848 - make sure group names are valid identifiers Fredrik Lundh <fredrik@pythonware.com> parents: 13121 diff changeset	214 return "0" <= char <= "9"
53d00e7c5848 - make sure group names are valid identifiers Fredrik Lundh <fredrik@pythonware.com> parents: 13121 diff changeset	215
53d00e7c5848 - make sure group names are valid identifiers Fredrik Lundh <fredrik@pythonware.com> parents: 13121 diff changeset	216 def isname(name):
53d00e7c5848 - make sure group names are valid identifiers Fredrik Lundh <fredrik@pythonware.com> parents: 13121 diff changeset	217 # check that group name is a valid string
53d00e7c5848 - make sure group names are valid identifiers Fredrik Lundh <fredrik@pythonware.com> parents: 13121 diff changeset	218 if not isident(name[0]):
22867 e2f2187c3d2c Partial introduction of bools where appropriate. Guido van Rossum <guido@python.org> parents: 21925 diff changeset	219 return False
33397 98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	220 for char in name[1:]:
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	221 if not isident(char) and not isdigit(char):
22867 e2f2187c3d2c Partial introduction of bools where appropriate. Guido van Rossum <guido@python.org> parents: 21925 diff changeset	222 return False
e2f2187c3d2c Partial introduction of bools where appropriate. Guido van Rossum <guido@python.org> parents: 21925 diff changeset	223 return True
13122 53d00e7c5848 - make sure group names are valid identifiers Fredrik Lundh <fredrik@pythonware.com> parents: 13121 diff changeset	224
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	225 def _class_escape(source, escape):
9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	226 # handle escape code inside character class
9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	227 code = ESCAPES.get(escape)
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	228 if code:
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	229 return code
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	230 code = CATEGORIES.get(escape)
9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	231 if code:
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	232 return code
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	233 try:
33397 98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	234 c = escape[1:2]
98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	235 if c == "x":
14316 511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	236 # hexadecimal escape (exactly two digits)
511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	237 while source.next in HEXDIGITS and len(escape) < 4:
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	238 escape = escape + source.get()
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	239 escape = escape[2:]
14316 511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	240 if len(escape) != 2:
511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	241 raise error, "bogus escape: %s" % repr("\\" + escape)
33257 11a4fbc88a80 PEP 292 classes Template and SafeTemplate are added to the string module. Barry Warsaw <barry@python.org> parents: 31829 diff changeset	242 return LITERAL, int(escape, 16) & 0xff
33397 98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	243 elif c in OCTDIGITS:
14316 511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	244 # octal escape (up to three digits)
33397 98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	245 while source.next in OCTDIGITS and len(escape) < 4:
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	246 escape = escape + source.get()
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	247 escape = escape[1:]
33257 11a4fbc88a80 PEP 292 classes Template and SafeTemplate are added to the string module. Barry Warsaw <barry@python.org> parents: 31829 diff changeset	248 return LITERAL, int(escape, 8) & 0xff
33397 98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	249 elif c in DIGITS:
98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	250 raise error, "bogus escape: %s" % repr(escape)
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	251 if len(escape) == 2:
13201 a093fad92ccf the mad patcher strikes again: Fredrik Lundh <fredrik@pythonware.com> parents: 13198 diff changeset	252 return LITERAL, ord(escape[1])
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	253 except ValueError:
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	254 pass
13118 7ff40da08c2a towards 1.6b1 Fredrik Lundh <fredrik@pythonware.com> parents: 12903 diff changeset	255 raise error, "bogus escape: %s" % repr(escape)
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	256
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	257 def _escape(source, escape, state):
9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	258 # handle escape code in expression
9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	259 code = CATEGORIES.get(escape)
9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	260 if code:
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	261 return code
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	262 code = ESCAPES.get(escape)
9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	263 if code:
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	264 return code
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	265 try:
33397 98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	266 c = escape[1:2]
98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	267 if c == "x":
14316 511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	268 # hexadecimal escape
511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	269 while source.next in HEXDIGITS and len(escape) < 4:
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	270 escape = escape + source.get()
14379 b069d7d9ce8a -- tightened up parsing of octal numbers Fredrik Lundh <fredrik@pythonware.com> parents: 14376 diff changeset	271 if len(escape) != 4:
b069d7d9ce8a -- tightened up parsing of octal numbers Fredrik Lundh <fredrik@pythonware.com> parents: 14376 diff changeset	272 raise ValueError
33257 11a4fbc88a80 PEP 292 classes Template and SafeTemplate are added to the string module. Barry Warsaw <barry@python.org> parents: 31829 diff changeset	273 return LITERAL, int(escape[2:], 16) & 0xff
33397 98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	274 elif c == "0":
14316 511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	275 # octal escape
14379 b069d7d9ce8a -- tightened up parsing of octal numbers Fredrik Lundh <fredrik@pythonware.com> parents: 14376 diff changeset	276 while source.next in OCTDIGITS and len(escape) < 4:
14316 511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	277 escape = escape + source.get()
33257 11a4fbc88a80 PEP 292 classes Template and SafeTemplate are added to the string module. Barry Warsaw <barry@python.org> parents: 31829 diff changeset	278 return LITERAL, int(escape[1:], 8) & 0xff
33397 98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	279 elif c in DIGITS:
14316 511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	280 # octal escape or decimal group reference (sigh)
511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	281 if source.next in DIGITS:
511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	282 escape = escape + source.get()
14379 b069d7d9ce8a -- tightened up parsing of octal numbers Fredrik Lundh <fredrik@pythonware.com> parents: 14376 diff changeset	283 if (escape[1] in OCTDIGITS and escape[2] in OCTDIGITS and
b069d7d9ce8a -- tightened up parsing of octal numbers Fredrik Lundh <fredrik@pythonware.com> parents: 14376 diff changeset	284 source.next in OCTDIGITS):
14316 511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	285 # got three octal digits; this is an octal escape
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	286 escape = escape + source.get()
33257 11a4fbc88a80 PEP 292 classes Template and SafeTemplate are added to the string module. Barry Warsaw <barry@python.org> parents: 31829 diff changeset	287 return LITERAL, int(escape[1:], 8) & 0xff
33397 98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	288 # not an octal escape, so this is a group reference
98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	289 group = int(escape[1:])
98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	290 if group < state.groups:
15355 2a3696d32721 -- properly reset groups in findall (bug #117612) Fredrik Lundh <fredrik@pythonware.com> parents: 15091 diff changeset	291 if not state.checkgroup(group):
2a3696d32721 -- properly reset groups in findall (bug #117612) Fredrik Lundh <fredrik@pythonware.com> parents: 15091 diff changeset	292 raise error, "cannot refer to open group"
14316 511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	293 return GROUPREF, group
14379 b069d7d9ce8a -- tightened up parsing of octal numbers Fredrik Lundh <fredrik@pythonware.com> parents: 14376 diff changeset	294 raise ValueError
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	295 if len(escape) == 2:
13201 a093fad92ccf the mad patcher strikes again: Fredrik Lundh <fredrik@pythonware.com> parents: 13198 diff changeset	296 return LITERAL, ord(escape[1])
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	297 except ValueError:
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	298 pass
13118 7ff40da08c2a towards 1.6b1 Fredrik Lundh <fredrik@pythonware.com> parents: 12903 diff changeset	299 raise error, "bogus escape: %s" % repr(escape)
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	300
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	301 def _parse_sub(source, state, nested=1):
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	302 # parse an alternation: a\|b\|c
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	303
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	304 items = []
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	305 itemsappend = items.append
a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	306 sourcematch = source.match
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	307 while 1:
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	308 itemsappend(_parse(source, state))
a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	309 if sourcematch("\|"):
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	310 continue
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	311 if not nested:
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	312 break
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	313 if not source.next or sourcematch(")", 0):
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	314 break
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	315 else:
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	316 raise error, "pattern not properly closed"
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	317
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	318 if len(items) == 1:
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	319 return items[0]
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	320
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	321 subpattern = SubPattern(state)
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	322 subpatternappend = subpattern.append
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	323
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	324 # check if all items share a common prefix
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	325 while 1:
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	326 prefix = None
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	327 for item in items:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	328 if not item:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	329 break
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	330 if prefix is None:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	331 prefix = item[0]
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	332 elif item[0] != prefix:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	333 break
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	334 else:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	335 # all subitems start with a common "prefix".
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	336 # move it out of the branch
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	337 for item in items:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	338 del item[0]
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	339 subpatternappend(prefix)
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	340 continue # check next one
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	341 break
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	342
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	343 # check if the branch can be replaced by a character set
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	344 for item in items:
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	345 if len(item) != 1 or item[0][0] != LITERAL:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	346 break
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	347 else:
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	348 # we can store this as a character set instead of a
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	349 # branch (the compiler may optimize this even more)
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	350 set = []
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	351 setappend = set.append
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	352 for item in items:
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	353 setappend(item[0])
a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	354 subpatternappend((IN, set))
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	355 return subpattern
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	356
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	357 subpattern.append((BRANCH, (None, items)))
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	358 return subpattern
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	359
30703 8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	360 def _parse_sub_cond(source, state, condgroup):
31389 aca1ce1b4b61 Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 30703 diff changeset	361 item_yes = _parse(source, state)
30703 8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	362 if source.match("\|"):
31389 aca1ce1b4b61 Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 30703 diff changeset	363 item_no = _parse(source, state)
30703 8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	364 if source.match("\|"):
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	365 raise error, "conditional backref with more than two branches"
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	366 else:
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	367 item_no = None
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	368 if source.next and not source.match(")", 0):
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	369 raise error, "pattern not properly closed"
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	370 subpattern = SubPattern(state)
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	371 subpattern.append((GROUPREF_EXISTS, (condgroup, item_yes, item_no)))
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	372 return subpattern
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	373
34635 5f26298811b7 Complete the previous effort to factor out constant expressions Raymond Hettinger <python@rcn.com> parents: 33400 diff changeset	374 _PATTERNENDERS = set("\|)")
5f26298811b7 Complete the previous effort to factor out constant expressions Raymond Hettinger <python@rcn.com> parents: 33400 diff changeset	375 _ASSERTCHARS = set("=!<")
5f26298811b7 Complete the previous effort to factor out constant expressions Raymond Hettinger <python@rcn.com> parents: 33400 diff changeset	376 _LOOKBEHINDASSERTCHARS = set("=!")
5f26298811b7 Complete the previous effort to factor out constant expressions Raymond Hettinger <python@rcn.com> parents: 33400 diff changeset	377 _REPEATCODES = set([MIN_REPEAT, MAX_REPEAT])
5f26298811b7 Complete the previous effort to factor out constant expressions Raymond Hettinger <python@rcn.com> parents: 33400 diff changeset	378
13250 88640ecf5506 - fixed code generation error in multiline mode Fredrik Lundh <fredrik@pythonware.com> parents: 13236 diff changeset	379 def _parse(source, state):
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	380 # parse a simple pattern
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	381 subpattern = SubPattern(state)
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	382
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	383 # precompute constants into local variables
a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	384 subpatternappend = subpattern.append
a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	385 sourceget = source.get
a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	386 sourcematch = source.match
a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	387 _len = len
34635 5f26298811b7 Complete the previous effort to factor out constant expressions Raymond Hettinger <python@rcn.com> parents: 33400 diff changeset	388 PATTERNENDERS = _PATTERNENDERS
5f26298811b7 Complete the previous effort to factor out constant expressions Raymond Hettinger <python@rcn.com> parents: 33400 diff changeset	389 ASSERTCHARS = _ASSERTCHARS
5f26298811b7 Complete the previous effort to factor out constant expressions Raymond Hettinger <python@rcn.com> parents: 33400 diff changeset	390 LOOKBEHINDASSERTCHARS = _LOOKBEHINDASSERTCHARS
5f26298811b7 Complete the previous effort to factor out constant expressions Raymond Hettinger <python@rcn.com> parents: 33400 diff changeset	391 REPEATCODES = _REPEATCODES
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	392
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	393 while 1:
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	394
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	395 if source.next in PATTERNENDERS:
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	396 break # end of subpattern
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	397 this = sourceget()
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	398 if this is None:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	399 break # end of pattern
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	400
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	401 if state.flags & SRE_FLAG_VERBOSE:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	402 # skip whitespace and comments
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	403 if this in WHITESPACE:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	404 continue
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	405 if this == "#":
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	406 while 1:
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	407 this = sourceget()
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	408 if this in (None, "\n"):
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	409 break
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	410 continue
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	411
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	412 if this and this[0] not in SPECIAL_CHARS:
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	413 subpatternappend((LITERAL, ord(this)))
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	414
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	415 elif this == "[":
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	416 # character set
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	417 set = []
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	418 setappend = set.append
a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	419 ## if sourcematch(":"):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	420 ## pass # handle character classes
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	421 if sourcematch("^"):
a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	422 setappend((NEGATE, None))
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	423 # check remaining characters
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	424 start = set[:]
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	425 while 1:
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	426 this = sourceget()
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	427 if this == "]" and set != start:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	428 break
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	429 elif this and this[0] == "\\":
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	430 code1 = _class_escape(source, this)
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	431 elif this:
13201 a093fad92ccf the mad patcher strikes again: Fredrik Lundh <fredrik@pythonware.com> parents: 13198 diff changeset	432 code1 = LITERAL, ord(this)
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	433 else:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	434 raise error, "unexpected end of regular expression"
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	435 if sourcematch("-"):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	436 # potential range
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	437 this = sourceget()
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	438 if this == "]":
15083 02ad9cb52222 SRE didn't handle character category followed by hyphen inside a Fredrik Lundh <fredrik@pythonware.com> parents: 14999 diff changeset	439 if code1[0] is IN:
02ad9cb52222 SRE didn't handle character category followed by hyphen inside a Fredrik Lundh <fredrik@pythonware.com> parents: 14999 diff changeset	440 code1 = code1[1][0]
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	441 setappend(code1)
a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	442 setappend((LITERAL, ord("-")))
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	443 break
28603 7fe33c1541df SF patch #720991 by Gary Herron: Guido van Rossum <guido@python.org> parents: 23627 diff changeset	444 elif this:
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	445 if this[0] == "\\":
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	446 code2 = _class_escape(source, this)
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	447 else:
13201 a093fad92ccf the mad patcher strikes again: Fredrik Lundh <fredrik@pythonware.com> parents: 13198 diff changeset	448 code2 = LITERAL, ord(this)
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	449 if code1[0] != LITERAL or code2[0] != LITERAL:
15858 81e3d8626adf SRE: stricter pattern syntax checking (covers parts of bug #115900) Fredrik Lundh <fredrik@pythonware.com> parents: 15855 diff changeset	450 raise error, "bad character range"
14316 511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	451 lo = code1[1]
511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	452 hi = code2[1]
511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	453 if hi < lo:
15858 81e3d8626adf SRE: stricter pattern syntax checking (covers parts of bug #115900) Fredrik Lundh <fredrik@pythonware.com> parents: 15855 diff changeset	454 raise error, "bad character range"
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	455 setappend((RANGE, (lo, hi)))
28603 7fe33c1541df SF patch #720991 by Gary Herron: Guido van Rossum <guido@python.org> parents: 23627 diff changeset	456 else:
7fe33c1541df SF patch #720991 by Gary Herron: Guido van Rossum <guido@python.org> parents: 23627 diff changeset	457 raise error, "unexpected end of regular expression"
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	458 else:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	459 if code1[0] is IN:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	460 code1 = code1[1][0]
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	461 setappend(code1)
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	462
15855 152d00718006 SRE fixes for 2.1 alpha: Fredrik Lundh <fredrik@pythonware.com> parents: 15355 diff changeset	463 # XXX: <fl> should move set optimization to compiler!
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	464 if _len(set)==1 and set[0][0] is LITERAL:
a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	465 subpatternappend(set[0]) # optimization
a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	466 elif _len(set)==2 and set[0][0] is NEGATE and set[1][0] is LITERAL:
a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	467 subpatternappend((NOT_LITERAL, set[1][1])) # optimization
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	468 else:
15855 152d00718006 SRE fixes for 2.1 alpha: Fredrik Lundh <fredrik@pythonware.com> parents: 15355 diff changeset	469 # XXX: <fl> should add charmap optimization here
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	470 subpatternappend((IN, set))
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	471
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	472 elif this and this[0] in REPEAT_CHARS:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	473 # repeat previous item
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	474 if this == "?":
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	475 min, max = 0, 1
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	476 elif this == "*":
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	477 min, max = 0, MAXREPEAT
16665 de0e8752d5b7 detect attempts to repeat anchors (fixes bug #130748) Fredrik Lundh <fredrik@pythonware.com> parents: 16663 diff changeset	478
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	479 elif this == "+":
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	480 min, max = 1, MAXREPEAT
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	481 elif this == "{":
35544 acfc0e22cd8f Bug #1202493: Fixing SRE parser to handle '{}' as perl does, rather than Gustavo Niemeyer <gustavo@niemeyer.net> parents: 34635 diff changeset	482 if source.next == "}":
acfc0e22cd8f Bug #1202493: Fixing SRE parser to handle '{}' as perl does, rather than Gustavo Niemeyer <gustavo@niemeyer.net> parents: 34635 diff changeset	483 subpatternappend((LITERAL, ord(this)))
acfc0e22cd8f Bug #1202493: Fixing SRE parser to handle '{}' as perl does, rather than Gustavo Niemeyer <gustavo@niemeyer.net> parents: 34635 diff changeset	484 continue
13305 33ce63b06359 - fixed "{ in any other context" bug Fredrik Lundh <fredrik@pythonware.com> parents: 13299 diff changeset	485 here = source.tell()
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	486 min, max = 0, MAXREPEAT
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	487 lo = hi = ""
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	488 while source.next in DIGITS:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	489 lo = lo + source.get()
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	490 if sourcematch(","):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	491 while source.next in DIGITS:
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	492 hi = hi + sourceget()
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	493 else:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	494 hi = lo
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	495 if not sourcematch("}"):
a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	496 subpatternappend((LITERAL, ord(this)))
13305 33ce63b06359 - fixed "{ in any other context" bug Fredrik Lundh <fredrik@pythonware.com> parents: 13299 diff changeset	497 source.seek(here)
33ce63b06359 - fixed "{ in any other context" bug Fredrik Lundh <fredrik@pythonware.com> parents: 13299 diff changeset	498 continue
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	499 if lo:
33257 11a4fbc88a80 PEP 292 classes Template and SafeTemplate are added to the string module. Barry Warsaw <barry@python.org> parents: 31829 diff changeset	500 min = int(lo)
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	501 if hi:
33257 11a4fbc88a80 PEP 292 classes Template and SafeTemplate are added to the string module. Barry Warsaw <barry@python.org> parents: 31829 diff changeset	502 max = int(hi)
15858 81e3d8626adf SRE: stricter pattern syntax checking (covers parts of bug #115900) Fredrik Lundh <fredrik@pythonware.com> parents: 15855 diff changeset	503 if max < min:
81e3d8626adf SRE: stricter pattern syntax checking (covers parts of bug #115900) Fredrik Lundh <fredrik@pythonware.com> parents: 15855 diff changeset	504 raise error, "bad repeat interval"
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	505 else:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	506 raise error, "not supported"
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	507 # figure out which item to repeat
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	508 if subpattern:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	509 item = subpattern[-1:]
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	510 else:
16665 de0e8752d5b7 detect attempts to repeat anchors (fixes bug #130748) Fredrik Lundh <fredrik@pythonware.com> parents: 16663 diff changeset	511 item = None
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	512 if not item or (_len(item) == 1 and item[0][0] == AT):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	513 raise error, "nothing to repeat"
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	514 if item[0][0] in REPEATCODES:
15858 81e3d8626adf SRE: stricter pattern syntax checking (covers parts of bug #115900) Fredrik Lundh <fredrik@pythonware.com> parents: 15855 diff changeset	515 raise error, "multiple repeat"
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	516 if sourcematch("?"):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	517 subpattern[-1] = (MIN_REPEAT, (min, max, item))
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	518 else:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	519 subpattern[-1] = (MAX_REPEAT, (min, max, item))
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	520
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	521 elif this == ".":
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	522 subpatternappend((ANY, None))
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	523
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	524 elif this == "(":
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	525 group = 1
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	526 name = None
30703 8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	527 condgroup = None
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	528 if sourcematch("?"):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	529 group = 0
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	530 # options
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	531 if sourcematch("P"):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	532 # python extensions
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	533 if sourcematch("<"):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	534 # named group: skip forward to end of name
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	535 name = ""
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	536 while 1:
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	537 char = sourceget()
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	538 if char is None:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	539 raise error, "unterminated name"
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	540 if char == ">":
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	541 break
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	542 name = name + char
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	543 group = 1
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	544 if not isname(name):
15858 81e3d8626adf SRE: stricter pattern syntax checking (covers parts of bug #115900) Fredrik Lundh <fredrik@pythonware.com> parents: 15855 diff changeset	545 raise error, "bad character in group name"
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	546 elif sourcematch("="):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	547 # named backreference
13188 92ba928c09b8 - added support for (?P=name) Fredrik Lundh <fredrik@pythonware.com> parents: 13186 diff changeset	548 name = ""
92ba928c09b8 - added support for (?P=name) Fredrik Lundh <fredrik@pythonware.com> parents: 13186 diff changeset	549 while 1:
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	550 char = sourceget()
13188 92ba928c09b8 - added support for (?P=name) Fredrik Lundh <fredrik@pythonware.com> parents: 13186 diff changeset	551 if char is None:
92ba928c09b8 - added support for (?P=name) Fredrik Lundh <fredrik@pythonware.com> parents: 13186 diff changeset	552 raise error, "unterminated name"
92ba928c09b8 - added support for (?P=name) Fredrik Lundh <fredrik@pythonware.com> parents: 13186 diff changeset	553 if char == ")":
92ba928c09b8 - added support for (?P=name) Fredrik Lundh <fredrik@pythonware.com> parents: 13186 diff changeset	554 break
92ba928c09b8 - added support for (?P=name) Fredrik Lundh <fredrik@pythonware.com> parents: 13186 diff changeset	555 name = name + char
92ba928c09b8 - added support for (?P=name) Fredrik Lundh <fredrik@pythonware.com> parents: 13186 diff changeset	556 if not isname(name):
15858 81e3d8626adf SRE: stricter pattern syntax checking (covers parts of bug #115900) Fredrik Lundh <fredrik@pythonware.com> parents: 15855 diff changeset	557 raise error, "bad character in group name"
13188 92ba928c09b8 - added support for (?P=name) Fredrik Lundh <fredrik@pythonware.com> parents: 13186 diff changeset	558 gid = state.groupdict.get(name)
92ba928c09b8 - added support for (?P=name) Fredrik Lundh <fredrik@pythonware.com> parents: 13186 diff changeset	559 if gid is None:
92ba928c09b8 - added support for (?P=name) Fredrik Lundh <fredrik@pythonware.com> parents: 13186 diff changeset	560 raise error, "unknown group name"
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	561 subpatternappend((GROUPREF, gid))
13308 fc2ad6fa08ec - actually enabled charset anchors in the engine (still not Fredrik Lundh <fredrik@pythonware.com> parents: 13306 diff changeset	562 continue
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	563 else:
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	564 char = sourceget()
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	565 if char is None:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	566 raise error, "unexpected end of pattern"
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	567 raise error, "unknown specifier: ?P%s" % char
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	568 elif sourcematch(":"):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	569 # non-capturing group
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	570 group = 2
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	571 elif sourcematch("#"):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	572 # comment
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	573 while 1:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	574 if source.next is None or source.next == ")":
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	575 break
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	576 sourceget()
a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	577 if not sourcematch(")"):
14316 511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	578 raise error, "unbalanced parenthesis"
511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	579 continue
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	580 elif source.next in ASSERTCHARS:
13198 4deeed58bf76 - fixed lookahead assertions (#10, #11, #12) Fredrik Lundh <fredrik@pythonware.com> parents: 13188 diff changeset	581 # lookahead assertions
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	582 char = sourceget()
13327 c155bccb7a22 - added lookbehind support (?<=pattern), (?<!pattern). Fredrik Lundh <fredrik@pythonware.com> parents: 13310 diff changeset	583 dir = 1
c155bccb7a22 - added lookbehind support (?<=pattern), (?<!pattern). Fredrik Lundh <fredrik@pythonware.com> parents: 13310 diff changeset	584 if char == "<":
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	585 if source.next not in LOOKBEHINDASSERTCHARS:
13327 c155bccb7a22 - added lookbehind support (?<=pattern), (?<!pattern). Fredrik Lundh <fredrik@pythonware.com> parents: 13310 diff changeset	586 raise error, "syntax error"
c155bccb7a22 - added lookbehind support (?<=pattern), (?<!pattern). Fredrik Lundh <fredrik@pythonware.com> parents: 13310 diff changeset	587 dir = -1 # lookbehind
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	588 char = sourceget()
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	589 p = _parse_sub(source, state)
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	590 if not sourcematch(")"):
14316 511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	591 raise error, "unbalanced parenthesis"
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	592 if char == "=":
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	593 subpatternappend((ASSERT, (dir, p)))
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	594 else:
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	595 subpatternappend((ASSERT_NOT, (dir, p)))
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	596 continue
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	597 elif sourcematch("("):
30703 8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	598 # conditional backreference group
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	599 condname = ""
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	600 while 1:
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	601 char = sourceget()
30703 8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	602 if char is None:
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	603 raise error, "unterminated name"
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	604 if char == ")":
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	605 break
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	606 condname = condname + char
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	607 group = 2
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	608 if isname(condname):
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	609 condgroup = state.groupdict.get(condname)
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	610 if condgroup is None:
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	611 raise error, "unknown group name"
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	612 else:
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	613 try:
33257 11a4fbc88a80 PEP 292 classes Template and SafeTemplate are added to the string module. Barry Warsaw <barry@python.org> parents: 31829 diff changeset	614 condgroup = int(condname)
30703 8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	615 except ValueError:
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	616 raise error, "bad character in group name"
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	617 else:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	618 # flags
23620 a1de0bab5475 SF 563203. Replaced 'has_key()' with 'in'. Raymond Hettinger <python@rcn.com> parents: 22867 diff changeset	619 if not source.next in FLAGS:
15858 81e3d8626adf SRE: stricter pattern syntax checking (covers parts of bug #115900) Fredrik Lundh <fredrik@pythonware.com> parents: 15855 diff changeset	620 raise error, "unexpected end of pattern"
23620 a1de0bab5475 SF 563203. Replaced 'has_key()' with 'in'. Raymond Hettinger <python@rcn.com> parents: 22867 diff changeset	621 while source.next in FLAGS:
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	622 state.flags = state.flags \| FLAGS[sourceget()]
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	623 if group:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	624 # parse group contents
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	625 if group == 2:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	626 # anonymous group
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	627 group = None
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	628 else:
15355 2a3696d32721 -- properly reset groups in findall (bug #117612) Fredrik Lundh <fredrik@pythonware.com> parents: 15091 diff changeset	629 group = state.opengroup(name)
30703 8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	630 if condgroup:
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	631 p = _parse_sub_cond(source, state, condgroup)
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	632 else:
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	633 p = _parse_sub(source, state)
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	634 if not sourcematch(")"):
14316 511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	635 raise error, "unbalanced parenthesis"
15355 2a3696d32721 -- properly reset groups in findall (bug #117612) Fredrik Lundh <fredrik@pythonware.com> parents: 15091 diff changeset	636 if group is not None:
2a3696d32721 -- properly reset groups in findall (bug #117612) Fredrik Lundh <fredrik@pythonware.com> parents: 15091 diff changeset	637 state.closegroup(group)
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	638 subpatternappend((SUBPATTERN, (group, p)))
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	639 else:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	640 while 1:
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	641 char = sourceget()
15858 81e3d8626adf SRE: stricter pattern syntax checking (covers parts of bug #115900) Fredrik Lundh <fredrik@pythonware.com> parents: 15855 diff changeset	642 if char is None:
81e3d8626adf SRE: stricter pattern syntax checking (covers parts of bug #115900) Fredrik Lundh <fredrik@pythonware.com> parents: 15855 diff changeset	643 raise error, "unexpected end of pattern"
81e3d8626adf SRE: stricter pattern syntax checking (covers parts of bug #115900) Fredrik Lundh <fredrik@pythonware.com> parents: 15855 diff changeset	644 if char == ")":
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	645 break
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	646 raise error, "unknown extension"
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	647
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	648 elif this == "^":
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	649 subpatternappend((AT, AT_BEGINNING))
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	650
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	651 elif this == "$":
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	652 subpattern.append((AT, AT_END))
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	653
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	654 elif this and this[0] == "\\":
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	655 code = _escape(source, this, state)
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	656 subpatternappend(code)
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	657
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	658 else:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	659 raise error, "parser error"
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	660
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	661 return subpattern
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	662
13886 62494a96dcca -- reset marks if repeat_one tail doesn't match Fredrik Lundh <fredrik@pythonware.com> parents: 13795 diff changeset	663 def parse(str, flags=0, pattern=None):
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	664 # parse 're' pattern into list of (opcode, argument) tuples
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	665
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	666 source = Tokenizer(str)
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	667
13886 62494a96dcca -- reset marks if repeat_one tail doesn't match Fredrik Lundh <fredrik@pythonware.com> parents: 13795 diff changeset	668 if pattern is None:
62494a96dcca -- reset marks if repeat_one tail doesn't match Fredrik Lundh <fredrik@pythonware.com> parents: 13795 diff changeset	669 pattern = Pattern()
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	670 pattern.flags = flags
15858 81e3d8626adf SRE: stricter pattern syntax checking (covers parts of bug #115900) Fredrik Lundh <fredrik@pythonware.com> parents: 15855 diff changeset	671 pattern.str = str
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	672
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	673 p = _parse_sub(source, pattern, 0)
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	674
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	675 tail = source.get()
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	676 if tail == ")":
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	677 raise error, "unbalanced parenthesis"
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	678 elif tail:
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	679 raise error, "bogus characters at end of regular expression"
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	680
15855 152d00718006 SRE fixes for 2.1 alpha: Fredrik Lundh <fredrik@pythonware.com> parents: 15355 diff changeset	681 if flags & SRE_FLAG_DEBUG:
152d00718006 SRE fixes for 2.1 alpha: Fredrik Lundh <fredrik@pythonware.com> parents: 15355 diff changeset	682 p.dump()
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	683
14999 1a3b5c8d6730 Recompile pattern if (?x) flag was found inside the pattern during the Fredrik Lundh <fredrik@pythonware.com> parents: 14797 diff changeset	684 if not (flags & SRE_FLAG_VERBOSE) and p.pattern.flags & SRE_FLAG_VERBOSE:
1a3b5c8d6730 Recompile pattern if (?x) flag was found inside the pattern during the Fredrik Lundh <fredrik@pythonware.com> parents: 14797 diff changeset	685 # the VERBOSE flag was switched on inside the pattern. to be
1a3b5c8d6730 Recompile pattern if (?x) flag was found inside the pattern during the Fredrik Lundh <fredrik@pythonware.com> parents: 14797 diff changeset	686 # on the safe side, we'll parse the whole thing again...
1a3b5c8d6730 Recompile pattern if (?x) flag was found inside the pattern during the Fredrik Lundh <fredrik@pythonware.com> parents: 14797 diff changeset	687 return parse(str, p.pattern.flags)
1a3b5c8d6730 Recompile pattern if (?x) flag was found inside the pattern during the Fredrik Lundh <fredrik@pythonware.com> parents: 14797 diff changeset	688
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	689 return p
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	690
13118 7ff40da08c2a towards 1.6b1 Fredrik Lundh <fredrik@pythonware.com> parents: 12903 diff changeset	691 def parse_template(source, pattern):
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	692 # parse 're' replacement string into list of literals and
9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	693 # group references
9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	694 s = Tokenizer(source)
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	695 sget = s.get
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	696 p = []
9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	697 a = p.append
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	698 def literal(literal, p=p, pappend=a):
17144 bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	699 if p and p[-1][0] is LITERAL:
bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	700 p[-1] = LITERAL, p[-1][1] + literal
bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	701 else:
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	702 pappend((LITERAL, literal))
17144 bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	703 sep = source[:0]
bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	704 if type(sep) is type(""):
19971 d4c965316610 fixed #449964: sre.sub raises an exception if the template contains a Fredrik Lundh <fredrik@pythonware.com> parents: 19693 diff changeset	705 makechar = chr
17144 bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	706 else:
19971 d4c965316610 fixed #449964: sre.sub raises an exception if the template contains a Fredrik Lundh <fredrik@pythonware.com> parents: 19693 diff changeset	707 makechar = unichr
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	708 while 1:
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	709 this = sget()
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	710 if this is None:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	711 break # end of replacement string
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	712 if this and this[0] == "\\":
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	713 # group
33397 98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	714 c = this[1:2]
98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	715 if c == "g":
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	716 name = ""
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	717 if s.match("<"):
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	718 while 1:
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	719 char = sget()
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	720 if char is None:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	721 raise error, "unterminated group name"
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	722 if char == ">":
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	723 break
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	724 name = name + char
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	725 if not name:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	726 raise error, "bad group name"
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	727 try:
33257 11a4fbc88a80 PEP 292 classes Template and SafeTemplate are added to the string module. Barry Warsaw <barry@python.org> parents: 31829 diff changeset	728 index = int(name)
33397 98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	729 if index < 0:
98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	730 raise error, "negative group number"
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	731 except ValueError:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	732 if not isname(name):
15858 81e3d8626adf SRE: stricter pattern syntax checking (covers parts of bug #115900) Fredrik Lundh <fredrik@pythonware.com> parents: 15855 diff changeset	733 raise error, "bad character in group name"
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	734 try:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	735 index = pattern.groupindex[name]
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	736 except KeyError:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	737 raise IndexError, "unknown group name"
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	738 a((MARK, index))
33397 98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	739 elif c == "0":
98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	740 if s.next in OCTDIGITS:
98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	741 this = this + sget()
98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	742 if s.next in OCTDIGITS:
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	743 this = this + sget()
33397 98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	744 literal(makechar(int(this[1:], 8) & 0xff))
98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	745 elif c in DIGITS:
98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	746 isoctal = False
98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	747 if s.next in DIGITS:
98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	748 this = this + sget()
33400 b9945335da27 Minor aesthetic change in sre_parse.py, as suggested by Mike Coleman. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33397 diff changeset	749 if (c in OCTDIGITS and this[2] in OCTDIGITS and
b9945335da27 Minor aesthetic change in sre_parse.py, as suggested by Mike Coleman. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33397 diff changeset	750 s.next in OCTDIGITS):
33397 98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	751 this = this + sget()
98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	752 isoctal = True
98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	753 literal(makechar(int(this[1:], 8) & 0xff))
98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	754 if not isoctal:
98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	755 a((MARK, int(this[1:])))
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	756 else:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	757 try:
19971 d4c965316610 fixed #449964: sre.sub raises an exception if the template contains a Fredrik Lundh <fredrik@pythonware.com> parents: 19693 diff changeset	758 this = makechar(ESCAPES[this][1])
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	759 except KeyError:
17144 bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	760 pass
bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	761 literal(this)
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	762 else:
17144 bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	763 literal(this)
bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	764 # convert template to groups and literals lists
bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	765 i = 0
bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	766 groups = []
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	767 groupsappend = groups.append
a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	768 literals = [None] * len(p)
17144 bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	769 for c, s in p:
bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	770 if c is MARK:
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	771 groupsappend((i, s))
a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	772 # literal[i] is already None
17144 bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	773 else:
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	774 literals[i] = s
17144 bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	775 i = i + 1
bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	776 return groups, literals
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	777
13118 7ff40da08c2a towards 1.6b1 Fredrik Lundh <fredrik@pythonware.com> parents: 12903 diff changeset	778 def expand_template(template, match):
17144 bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	779 g = match.group
13201 a093fad92ccf the mad patcher strikes again: Fredrik Lundh <fredrik@pythonware.com> parents: 13198 diff changeset	780 sep = match.string[:0]
17144 bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	781 groups, literals = template
bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	782 literals = literals[:]
bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	783 try:
bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	784 for index, group in groups:
bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	785 literals[index] = s = g(group)
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	786 if s is None:
33397 98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	787 raise error, "unmatched group"
17144 bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	788 except IndexError:
33397 98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	789 raise error, "invalid group reference"
33257 11a4fbc88a80 PEP 292 classes Template and SafeTemplate are added to the string module. Barry Warsaw <barry@python.org> parents: 31829 diff changeset	790 return sep.join(literals)

Mercurial > cpython-lru_cache-weakref

annotate Lib/sre_parse.py @ 79675:c0a2294de787 2.7