cpython-lru_cache-weakref: Lib/sre

annotate Lib/sre_parse.py @ 58925:eecf1bef052d 2.6

Merged revisions 78207 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r78207 | ezio.melotti | 2010-02-17 01:26:09 +0200 (Wed, 17 Feb 2010) | 1 line #7930: fix stripid ........

author	Ezio Melotti <ezio.melotti@gmail.com>
date	Tue, 16 Feb 2010 23:29:44 +0000
parents	343aaaa802e8
children	b09e7053b472

rev	line source
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	1 #
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	2 # Secret Labs' Regular Expression Engine
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	3 #
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	4 # convert re-style regular expression to sre pattern
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	5 #
15855 152d00718006 SRE fixes for 2.1 alpha: Fredrik Lundh <fredrik@pythonware.com> parents: 15355 diff changeset	6 # Copyright (c) 1998-2001 by Secret Labs AB. All rights reserved.
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	7 #
13789 4edd13eb759d SRE 0.9.8: passes the entire test suite Fredrik Lundh <fredrik@pythonware.com> parents: 13677 diff changeset	8 # See the sre.py file for information on usage and redistribution.
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	9 #
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	10
19693 617f5e72b0ec Added docstrings by Neal Norwitz. This closes SF bug #450980. Fred Drake <fdrake@acm.org> parents: 17144 diff changeset	11 """Internal support module for sre"""
617f5e72b0ec Added docstrings by Neal Norwitz. This closes SF bug #450980. Fred Drake <fdrake@acm.org> parents: 17144 diff changeset	12
15858 81e3d8626adf SRE: stricter pattern syntax checking (covers parts of bug #115900) Fredrik Lundh <fredrik@pythonware.com> parents: 15855 diff changeset	13 # XXX: show string offset and offending character for all errors
81e3d8626adf SRE: stricter pattern syntax checking (covers parts of bug #115900) Fredrik Lundh <fredrik@pythonware.com> parents: 15855 diff changeset	14
33257 11a4fbc88a80 PEP 292 classes Template and SafeTemplate are added to the string module. Barry Warsaw <barry@python.org> parents: 31829 diff changeset	15 import sys
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	16
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	17 from sre_constants import *
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	18
34635 5f26298811b7 Complete the previous effort to factor out constant expressions Raymond Hettinger <python@rcn.com> parents: 33400 diff changeset	19 def set(seq):
5f26298811b7 Complete the previous effort to factor out constant expressions Raymond Hettinger <python@rcn.com> parents: 33400 diff changeset	20 s = {}
5f26298811b7 Complete the previous effort to factor out constant expressions Raymond Hettinger <python@rcn.com> parents: 33400 diff changeset	21 for elem in seq:
5f26298811b7 Complete the previous effort to factor out constant expressions Raymond Hettinger <python@rcn.com> parents: 33400 diff changeset	22 s[elem] = 1
5f26298811b7 Complete the previous effort to factor out constant expressions Raymond Hettinger <python@rcn.com> parents: 33400 diff changeset	23 return s
5f26298811b7 Complete the previous effort to factor out constant expressions Raymond Hettinger <python@rcn.com> parents: 33400 diff changeset	24
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	25 SPECIAL_CHARS = ".\\[{()*+?^$\|"
14379 b069d7d9ce8a -- tightened up parsing of octal numbers Fredrik Lundh <fredrik@pythonware.com> parents: 14376 diff changeset	26 REPEAT_CHARS = "*+?{"
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	27
34635 5f26298811b7 Complete the previous effort to factor out constant expressions Raymond Hettinger <python@rcn.com> parents: 33400 diff changeset	28 DIGITS = set("0123456789")
12339 9e9666f679f7 Fredrik Lundh: new snapshot. Mostly reindented. Guido van Rossum <guido@python.org> parents: 12213 diff changeset	29
34635 5f26298811b7 Complete the previous effort to factor out constant expressions Raymond Hettinger <python@rcn.com> parents: 33400 diff changeset	30 OCTDIGITS = set("01234567")
5f26298811b7 Complete the previous effort to factor out constant expressions Raymond Hettinger <python@rcn.com> parents: 33400 diff changeset	31 HEXDIGITS = set("0123456789abcdefABCDEF")
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	32
34635 5f26298811b7 Complete the previous effort to factor out constant expressions Raymond Hettinger <python@rcn.com> parents: 33400 diff changeset	33 WHITESPACE = set(" \t\n\r\v\f")
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	34
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	35 ESCAPES = {
16663 94a38443f99b - restored 1.5.2 compatibility (sorry, eric) Fredrik Lundh <fredrik@pythonware.com> parents: 16613 diff changeset	36 r"\a": (LITERAL, ord("\a")),
94a38443f99b - restored 1.5.2 compatibility (sorry, eric) Fredrik Lundh <fredrik@pythonware.com> parents: 16613 diff changeset	37 r"\b": (LITERAL, ord("\b")),
94a38443f99b - restored 1.5.2 compatibility (sorry, eric) Fredrik Lundh <fredrik@pythonware.com> parents: 16613 diff changeset	38 r"\f": (LITERAL, ord("\f")),
94a38443f99b - restored 1.5.2 compatibility (sorry, eric) Fredrik Lundh <fredrik@pythonware.com> parents: 16613 diff changeset	39 r"\n": (LITERAL, ord("\n")),
94a38443f99b - restored 1.5.2 compatibility (sorry, eric) Fredrik Lundh <fredrik@pythonware.com> parents: 16613 diff changeset	40 r"\r": (LITERAL, ord("\r")),
94a38443f99b - restored 1.5.2 compatibility (sorry, eric) Fredrik Lundh <fredrik@pythonware.com> parents: 16613 diff changeset	41 r"\t": (LITERAL, ord("\t")),
94a38443f99b - restored 1.5.2 compatibility (sorry, eric) Fredrik Lundh <fredrik@pythonware.com> parents: 16613 diff changeset	42 r"\v": (LITERAL, ord("\v")),
13201 a093fad92ccf the mad patcher strikes again: Fredrik Lundh <fredrik@pythonware.com> parents: 13198 diff changeset	43 r"\\": (LITERAL, ord("\\"))
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	44 }
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	45
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	46 CATEGORIES = {
15855 152d00718006 SRE fixes for 2.1 alpha: Fredrik Lundh <fredrik@pythonware.com> parents: 15355 diff changeset	47 r"\A": (AT, AT_BEGINNING_STRING), # start of string
13166 023a2018b96d - fixed split behaviour on empty matches Fredrik Lundh <fredrik@pythonware.com> parents: 13139 diff changeset	48 r"\b": (AT, AT_BOUNDARY),
023a2018b96d - fixed split behaviour on empty matches Fredrik Lundh <fredrik@pythonware.com> parents: 13139 diff changeset	49 r"\B": (AT, AT_NON_BOUNDARY),
023a2018b96d - fixed split behaviour on empty matches Fredrik Lundh <fredrik@pythonware.com> parents: 13139 diff changeset	50 r"\d": (IN, [(CATEGORY, CATEGORY_DIGIT)]),
023a2018b96d - fixed split behaviour on empty matches Fredrik Lundh <fredrik@pythonware.com> parents: 13139 diff changeset	51 r"\D": (IN, [(CATEGORY, CATEGORY_NOT_DIGIT)]),
023a2018b96d - fixed split behaviour on empty matches Fredrik Lundh <fredrik@pythonware.com> parents: 13139 diff changeset	52 r"\s": (IN, [(CATEGORY, CATEGORY_SPACE)]),
023a2018b96d - fixed split behaviour on empty matches Fredrik Lundh <fredrik@pythonware.com> parents: 13139 diff changeset	53 r"\S": (IN, [(CATEGORY, CATEGORY_NOT_SPACE)]),
023a2018b96d - fixed split behaviour on empty matches Fredrik Lundh <fredrik@pythonware.com> parents: 13139 diff changeset	54 r"\w": (IN, [(CATEGORY, CATEGORY_WORD)]),
023a2018b96d - fixed split behaviour on empty matches Fredrik Lundh <fredrik@pythonware.com> parents: 13139 diff changeset	55 r"\W": (IN, [(CATEGORY, CATEGORY_NOT_WORD)]),
15855 152d00718006 SRE fixes for 2.1 alpha: Fredrik Lundh <fredrik@pythonware.com> parents: 15355 diff changeset	56 r"\Z": (AT, AT_END_STRING), # end of string
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	57 }
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	58
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	59 FLAGS = {
13118 7ff40da08c2a towards 1.6b1 Fredrik Lundh <fredrik@pythonware.com> parents: 12903 diff changeset	60 # standard flags
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	61 "i": SRE_FLAG_IGNORECASE,
9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	62 "L": SRE_FLAG_LOCALE,
9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	63 "m": SRE_FLAG_MULTILINE,
9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	64 "s": SRE_FLAG_DOTALL,
13118 7ff40da08c2a towards 1.6b1 Fredrik Lundh <fredrik@pythonware.com> parents: 12903 diff changeset	65 "x": SRE_FLAG_VERBOSE,
7ff40da08c2a towards 1.6b1 Fredrik Lundh <fredrik@pythonware.com> parents: 12903 diff changeset	66 # extensions
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	67 "t": SRE_FLAG_TEMPLATE,
13118 7ff40da08c2a towards 1.6b1 Fredrik Lundh <fredrik@pythonware.com> parents: 12903 diff changeset	68 "u": SRE_FLAG_UNICODE,
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	69 }
9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	70
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	71 class Pattern:
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	72 # master pattern object. keeps track of global attributes
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	73 def __init__(self):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	74 self.flags = 0
15355 2a3696d32721 -- properly reset groups in findall (bug #117612) Fredrik Lundh <fredrik@pythonware.com> parents: 15091 diff changeset	75 self.open = []
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	76 self.groups = 1
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	77 self.groupdict = {}
15355 2a3696d32721 -- properly reset groups in findall (bug #117612) Fredrik Lundh <fredrik@pythonware.com> parents: 15091 diff changeset	78 def opengroup(self, name=None):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	79 gid = self.groups
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	80 self.groups = gid + 1
23627 a6a765c7d33b Replace boolean test with is None. Raymond Hettinger <python@rcn.com> parents: 23620 diff changeset	81 if name is not None:
20860 0caea5fefa5c Improved error msg when a symbolic group name is redefined. Added docs Tim Peters <tim.peters@gmail.com> parents: 20830 diff changeset	82 ogid = self.groupdict.get(name, None)
0caea5fefa5c Improved error msg when a symbolic group name is redefined. Added docs Tim Peters <tim.peters@gmail.com> parents: 20830 diff changeset	83 if ogid is not None:
21319 4bcfb9dc57e0 bug #133283, #477728, #483789, #490573 Fredrik Lundh <fredrik@pythonware.com> parents: 20860 diff changeset	84 raise error, ("redefinition of group name %s as group %d; "
4bcfb9dc57e0 bug #133283, #477728, #483789, #490573 Fredrik Lundh <fredrik@pythonware.com> parents: 20860 diff changeset	85 "was group %d" % (repr(name), gid, ogid))
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	86 self.groupdict[name] = gid
15355 2a3696d32721 -- properly reset groups in findall (bug #117612) Fredrik Lundh <fredrik@pythonware.com> parents: 15091 diff changeset	87 self.open.append(gid)
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	88 return gid
15355 2a3696d32721 -- properly reset groups in findall (bug #117612) Fredrik Lundh <fredrik@pythonware.com> parents: 15091 diff changeset	89 def closegroup(self, gid):
2a3696d32721 -- properly reset groups in findall (bug #117612) Fredrik Lundh <fredrik@pythonware.com> parents: 15091 diff changeset	90 self.open.remove(gid)
2a3696d32721 -- properly reset groups in findall (bug #117612) Fredrik Lundh <fredrik@pythonware.com> parents: 15091 diff changeset	91 def checkgroup(self, gid):
2a3696d32721 -- properly reset groups in findall (bug #117612) Fredrik Lundh <fredrik@pythonware.com> parents: 15091 diff changeset	92 return gid < self.groups and gid not in self.open
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	93
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	94 class SubPattern:
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	95 # a subpattern, in intermediate form
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	96 def __init__(self, pattern, data=None):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	97 self.pattern = pattern
23627 a6a765c7d33b Replace boolean test with is None. Raymond Hettinger <python@rcn.com> parents: 23620 diff changeset	98 if data is None:
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	99 data = []
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	100 self.data = data
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	101 self.width = None
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	102 def dump(self, level=0):
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	103 nl = 1
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	104 seqtypes = type(()), type([])
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	105 for op, av in self.data:
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	106 print level*" " + op,; nl = 0
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	107 if op == "in":
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	108 # member sublanguage
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	109 print; nl = 1
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	110 for op, a in av:
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	111 print (level+1)*" " + op, a
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	112 elif op == "branch":
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	113 print; nl = 1
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	114 i = 0
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	115 for a in av[1]:
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	116 if i > 0:
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	117 print level*" " + "or"
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	118 a.dump(level+1); nl = 1
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	119 i = i + 1
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	120 elif type(av) in seqtypes:
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	121 for a in av:
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	122 if isinstance(a, SubPattern):
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	123 if not nl: print
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	124 a.dump(level+1); nl = 1
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	125 else:
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	126 print a, ; nl = 0
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	127 else:
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	128 print av, ; nl = 0
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	129 if not nl: print
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	130 def __repr__(self):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	131 return repr(self.data)
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	132 def __len__(self):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	133 return len(self.data)
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	134 def __delitem__(self, index):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	135 del self.data[index]
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	136 def __getitem__(self, index):
40471 eb3b239a4d07 Make sre's SubPattern objects accept slice objects like it already accepts Thomas Wouters <thomas@python.org> parents: 35544 diff changeset	137 if isinstance(index, slice):
eb3b239a4d07 Make sre's SubPattern objects accept slice objects like it already accepts Thomas Wouters <thomas@python.org> parents: 35544 diff changeset	138 return SubPattern(self.pattern, self.data[index])
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	139 return self.data[index]
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	140 def __setitem__(self, index, code):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	141 self.data[index] = code
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	142 def insert(self, index, code):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	143 self.data.insert(index, code)
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	144 def append(self, code):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	145 self.data.append(code)
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	146 def getwidth(self):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	147 # determine the width (min, max) for this subpattern
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	148 if self.width:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	149 return self.width
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	150 lo = hi = 0L
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	151 UNITCODES = (ANY, RANGE, IN, LITERAL, NOT_LITERAL, CATEGORY)
a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	152 REPEATCODES = (MIN_REPEAT, MAX_REPEAT)
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	153 for op, av in self.data:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	154 if op is BRANCH:
13793 4c10bc811fed -- fixed width calculations for alternations Fredrik Lundh <fredrik@pythonware.com> parents: 13789 diff changeset	155 i = sys.maxint
4c10bc811fed -- fixed width calculations for alternations Fredrik Lundh <fredrik@pythonware.com> parents: 13789 diff changeset	156 j = 0
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	157 for av in av[1]:
13793 4c10bc811fed -- fixed width calculations for alternations Fredrik Lundh <fredrik@pythonware.com> parents: 13789 diff changeset	158 l, h = av.getwidth()
4c10bc811fed -- fixed width calculations for alternations Fredrik Lundh <fredrik@pythonware.com> parents: 13789 diff changeset	159 i = min(i, l)
13795 8295f8506eae final 0.9.8 updates: Fredrik Lundh <fredrik@pythonware.com> parents: 13793 diff changeset	160 j = max(j, h)
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	161 lo = lo + i
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	162 hi = hi + j
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	163 elif op is CALL:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	164 i, j = av.getwidth()
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	165 lo = lo + i
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	166 hi = hi + j
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	167 elif op is SUBPATTERN:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	168 i, j = av[1].getwidth()
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	169 lo = lo + i
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	170 hi = hi + j
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	171 elif op in REPEATCODES:
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	172 i, j = av[2].getwidth()
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	173 lo = lo + long(i) * av[0]
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	174 hi = hi + long(j) * av[1]
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	175 elif op in UNITCODES:
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	176 lo = lo + 1
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	177 hi = hi + 1
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	178 elif op == SUCCESS:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	179 break
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	180 self.width = int(min(lo, sys.maxint)), int(min(hi, sys.maxint))
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	181 return self.width
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	182
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	183 class Tokenizer:
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	184 def __init__(self, string):
13305 33ce63b06359 - fixed "{ in any other context" bug Fredrik Lundh <fredrik@pythonware.com> parents: 13299 diff changeset	185 self.string = string
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	186 self.index = 0
13305 33ce63b06359 - fixed "{ in any other context" bug Fredrik Lundh <fredrik@pythonware.com> parents: 13299 diff changeset	187 self.__next()
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	188 def __next(self):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	189 if self.index >= len(self.string):
13305 33ce63b06359 - fixed "{ in any other context" bug Fredrik Lundh <fredrik@pythonware.com> parents: 13299 diff changeset	190 self.next = None
33ce63b06359 - fixed "{ in any other context" bug Fredrik Lundh <fredrik@pythonware.com> parents: 13299 diff changeset	191 return
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	192 char = self.string[self.index]
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	193 if char[0] == "\\":
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	194 try:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	195 c = self.string[self.index + 1]
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	196 except IndexError:
20830 5f24b20e103d SF bug #476912: flag repeated use of the same groupname as Fredrik Lundh <fredrik@pythonware.com> parents: 19971 diff changeset	197 raise error, "bogus escape (end of line)"
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	198 char = char + c
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	199 self.index = self.index + len(char)
13305 33ce63b06359 - fixed "{ in any other context" bug Fredrik Lundh <fredrik@pythonware.com> parents: 13299 diff changeset	200 self.next = char
14316 511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	201 def match(self, char, skip=1):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	202 if char == self.next:
14316 511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	203 if skip:
511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	204 self.__next()
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	205 return 1
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	206 return 0
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	207 def get(self):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	208 this = self.next
13305 33ce63b06359 - fixed "{ in any other context" bug Fredrik Lundh <fredrik@pythonware.com> parents: 13299 diff changeset	209 self.__next()
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	210 return this
13305 33ce63b06359 - fixed "{ in any other context" bug Fredrik Lundh <fredrik@pythonware.com> parents: 13299 diff changeset	211 def tell(self):
33ce63b06359 - fixed "{ in any other context" bug Fredrik Lundh <fredrik@pythonware.com> parents: 13299 diff changeset	212 return self.index, self.next
33ce63b06359 - fixed "{ in any other context" bug Fredrik Lundh <fredrik@pythonware.com> parents: 13299 diff changeset	213 def seek(self, index):
33ce63b06359 - fixed "{ in any other context" bug Fredrik Lundh <fredrik@pythonware.com> parents: 13299 diff changeset	214 self.index, self.next = index
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	215
13122 53d00e7c5848 - make sure group names are valid identifiers Fredrik Lundh <fredrik@pythonware.com> parents: 13121 diff changeset	216 def isident(char):
53d00e7c5848 - make sure group names are valid identifiers Fredrik Lundh <fredrik@pythonware.com> parents: 13121 diff changeset	217 return "a" <= char <= "z" or "A" <= char <= "Z" or char == "_"
53d00e7c5848 - make sure group names are valid identifiers Fredrik Lundh <fredrik@pythonware.com> parents: 13121 diff changeset	218
53d00e7c5848 - make sure group names are valid identifiers Fredrik Lundh <fredrik@pythonware.com> parents: 13121 diff changeset	219 def isdigit(char):
53d00e7c5848 - make sure group names are valid identifiers Fredrik Lundh <fredrik@pythonware.com> parents: 13121 diff changeset	220 return "0" <= char <= "9"
53d00e7c5848 - make sure group names are valid identifiers Fredrik Lundh <fredrik@pythonware.com> parents: 13121 diff changeset	221
53d00e7c5848 - make sure group names are valid identifiers Fredrik Lundh <fredrik@pythonware.com> parents: 13121 diff changeset	222 def isname(name):
53d00e7c5848 - make sure group names are valid identifiers Fredrik Lundh <fredrik@pythonware.com> parents: 13121 diff changeset	223 # check that group name is a valid string
53d00e7c5848 - make sure group names are valid identifiers Fredrik Lundh <fredrik@pythonware.com> parents: 13121 diff changeset	224 if not isident(name[0]):
22867 e2f2187c3d2c Partial introduction of bools where appropriate. Guido van Rossum <guido@python.org> parents: 21925 diff changeset	225 return False
33397 98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	226 for char in name[1:]:
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	227 if not isident(char) and not isdigit(char):
22867 e2f2187c3d2c Partial introduction of bools where appropriate. Guido van Rossum <guido@python.org> parents: 21925 diff changeset	228 return False
e2f2187c3d2c Partial introduction of bools where appropriate. Guido van Rossum <guido@python.org> parents: 21925 diff changeset	229 return True
13122 53d00e7c5848 - make sure group names are valid identifiers Fredrik Lundh <fredrik@pythonware.com> parents: 13121 diff changeset	230
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	231 def _class_escape(source, escape):
9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	232 # handle escape code inside character class
9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	233 code = ESCAPES.get(escape)
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	234 if code:
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	235 return code
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	236 code = CATEGORIES.get(escape)
9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	237 if code:
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	238 return code
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	239 try:
33397 98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	240 c = escape[1:2]
98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	241 if c == "x":
14316 511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	242 # hexadecimal escape (exactly two digits)
511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	243 while source.next in HEXDIGITS and len(escape) < 4:
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	244 escape = escape + source.get()
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	245 escape = escape[2:]
14316 511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	246 if len(escape) != 2:
511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	247 raise error, "bogus escape: %s" % repr("\\" + escape)
33257 11a4fbc88a80 PEP 292 classes Template and SafeTemplate are added to the string module. Barry Warsaw <barry@python.org> parents: 31829 diff changeset	248 return LITERAL, int(escape, 16) & 0xff
33397 98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	249 elif c in OCTDIGITS:
14316 511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	250 # octal escape (up to three digits)
33397 98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	251 while source.next in OCTDIGITS and len(escape) < 4:
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	252 escape = escape + source.get()
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	253 escape = escape[1:]
33257 11a4fbc88a80 PEP 292 classes Template and SafeTemplate are added to the string module. Barry Warsaw <barry@python.org> parents: 31829 diff changeset	254 return LITERAL, int(escape, 8) & 0xff
33397 98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	255 elif c in DIGITS:
98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	256 raise error, "bogus escape: %s" % repr(escape)
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	257 if len(escape) == 2:
13201 a093fad92ccf the mad patcher strikes again: Fredrik Lundh <fredrik@pythonware.com> parents: 13198 diff changeset	258 return LITERAL, ord(escape[1])
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	259 except ValueError:
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	260 pass
13118 7ff40da08c2a towards 1.6b1 Fredrik Lundh <fredrik@pythonware.com> parents: 12903 diff changeset	261 raise error, "bogus escape: %s" % repr(escape)
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	262
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	263 def _escape(source, escape, state):
9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	264 # handle escape code in expression
9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	265 code = CATEGORIES.get(escape)
9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	266 if code:
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	267 return code
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	268 code = ESCAPES.get(escape)
9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	269 if code:
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	270 return code
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	271 try:
33397 98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	272 c = escape[1:2]
98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	273 if c == "x":
14316 511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	274 # hexadecimal escape
511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	275 while source.next in HEXDIGITS and len(escape) < 4:
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	276 escape = escape + source.get()
14379 b069d7d9ce8a -- tightened up parsing of octal numbers Fredrik Lundh <fredrik@pythonware.com> parents: 14376 diff changeset	277 if len(escape) != 4:
b069d7d9ce8a -- tightened up parsing of octal numbers Fredrik Lundh <fredrik@pythonware.com> parents: 14376 diff changeset	278 raise ValueError
33257 11a4fbc88a80 PEP 292 classes Template and SafeTemplate are added to the string module. Barry Warsaw <barry@python.org> parents: 31829 diff changeset	279 return LITERAL, int(escape[2:], 16) & 0xff
33397 98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	280 elif c == "0":
14316 511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	281 # octal escape
14379 b069d7d9ce8a -- tightened up parsing of octal numbers Fredrik Lundh <fredrik@pythonware.com> parents: 14376 diff changeset	282 while source.next in OCTDIGITS and len(escape) < 4:
14316 511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	283 escape = escape + source.get()
33257 11a4fbc88a80 PEP 292 classes Template and SafeTemplate are added to the string module. Barry Warsaw <barry@python.org> parents: 31829 diff changeset	284 return LITERAL, int(escape[1:], 8) & 0xff
33397 98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	285 elif c in DIGITS:
14316 511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	286 # octal escape or decimal group reference (sigh)
511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	287 if source.next in DIGITS:
511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	288 escape = escape + source.get()
14379 b069d7d9ce8a -- tightened up parsing of octal numbers Fredrik Lundh <fredrik@pythonware.com> parents: 14376 diff changeset	289 if (escape[1] in OCTDIGITS and escape[2] in OCTDIGITS and
b069d7d9ce8a -- tightened up parsing of octal numbers Fredrik Lundh <fredrik@pythonware.com> parents: 14376 diff changeset	290 source.next in OCTDIGITS):
14316 511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	291 # got three octal digits; this is an octal escape
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	292 escape = escape + source.get()
33257 11a4fbc88a80 PEP 292 classes Template and SafeTemplate are added to the string module. Barry Warsaw <barry@python.org> parents: 31829 diff changeset	293 return LITERAL, int(escape[1:], 8) & 0xff
33397 98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	294 # not an octal escape, so this is a group reference
98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	295 group = int(escape[1:])
98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	296 if group < state.groups:
15355 2a3696d32721 -- properly reset groups in findall (bug #117612) Fredrik Lundh <fredrik@pythonware.com> parents: 15091 diff changeset	297 if not state.checkgroup(group):
2a3696d32721 -- properly reset groups in findall (bug #117612) Fredrik Lundh <fredrik@pythonware.com> parents: 15091 diff changeset	298 raise error, "cannot refer to open group"
14316 511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	299 return GROUPREF, group
14379 b069d7d9ce8a -- tightened up parsing of octal numbers Fredrik Lundh <fredrik@pythonware.com> parents: 14376 diff changeset	300 raise ValueError
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	301 if len(escape) == 2:
13201 a093fad92ccf the mad patcher strikes again: Fredrik Lundh <fredrik@pythonware.com> parents: 13198 diff changeset	302 return LITERAL, ord(escape[1])
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	303 except ValueError:
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	304 pass
13118 7ff40da08c2a towards 1.6b1 Fredrik Lundh <fredrik@pythonware.com> parents: 12903 diff changeset	305 raise error, "bogus escape: %s" % repr(escape)
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	306
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	307 def _parse_sub(source, state, nested=1):
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	308 # parse an alternation: a\|b\|c
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	309
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	310 items = []
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	311 itemsappend = items.append
a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	312 sourcematch = source.match
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	313 while 1:
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	314 itemsappend(_parse(source, state))
a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	315 if sourcematch("\|"):
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	316 continue
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	317 if not nested:
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	318 break
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	319 if not source.next or sourcematch(")", 0):
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	320 break
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	321 else:
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	322 raise error, "pattern not properly closed"
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	323
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	324 if len(items) == 1:
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	325 return items[0]
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	326
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	327 subpattern = SubPattern(state)
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	328 subpatternappend = subpattern.append
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	329
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	330 # check if all items share a common prefix
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	331 while 1:
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	332 prefix = None
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	333 for item in items:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	334 if not item:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	335 break
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	336 if prefix is None:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	337 prefix = item[0]
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	338 elif item[0] != prefix:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	339 break
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	340 else:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	341 # all subitems start with a common "prefix".
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	342 # move it out of the branch
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	343 for item in items:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	344 del item[0]
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	345 subpatternappend(prefix)
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	346 continue # check next one
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	347 break
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	348
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	349 # check if the branch can be replaced by a character set
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	350 for item in items:
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	351 if len(item) != 1 or item[0][0] != LITERAL:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	352 break
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	353 else:
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	354 # we can store this as a character set instead of a
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	355 # branch (the compiler may optimize this even more)
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	356 set = []
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	357 setappend = set.append
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	358 for item in items:
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	359 setappend(item[0])
a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	360 subpatternappend((IN, set))
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	361 return subpattern
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	362
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	363 subpattern.append((BRANCH, (None, items)))
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	364 return subpattern
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	365
30703 8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	366 def _parse_sub_cond(source, state, condgroup):
31389 aca1ce1b4b61 Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 30703 diff changeset	367 item_yes = _parse(source, state)
30703 8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	368 if source.match("\|"):
31389 aca1ce1b4b61 Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 30703 diff changeset	369 item_no = _parse(source, state)
30703 8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	370 if source.match("\|"):
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	371 raise error, "conditional backref with more than two branches"
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	372 else:
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	373 item_no = None
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	374 if source.next and not source.match(")", 0):
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	375 raise error, "pattern not properly closed"
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	376 subpattern = SubPattern(state)
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	377 subpattern.append((GROUPREF_EXISTS, (condgroup, item_yes, item_no)))
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	378 return subpattern
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	379
34635 5f26298811b7 Complete the previous effort to factor out constant expressions Raymond Hettinger <python@rcn.com> parents: 33400 diff changeset	380 _PATTERNENDERS = set("\|)")
5f26298811b7 Complete the previous effort to factor out constant expressions Raymond Hettinger <python@rcn.com> parents: 33400 diff changeset	381 _ASSERTCHARS = set("=!<")
5f26298811b7 Complete the previous effort to factor out constant expressions Raymond Hettinger <python@rcn.com> parents: 33400 diff changeset	382 _LOOKBEHINDASSERTCHARS = set("=!")
5f26298811b7 Complete the previous effort to factor out constant expressions Raymond Hettinger <python@rcn.com> parents: 33400 diff changeset	383 _REPEATCODES = set([MIN_REPEAT, MAX_REPEAT])
5f26298811b7 Complete the previous effort to factor out constant expressions Raymond Hettinger <python@rcn.com> parents: 33400 diff changeset	384
13250 88640ecf5506 - fixed code generation error in multiline mode Fredrik Lundh <fredrik@pythonware.com> parents: 13236 diff changeset	385 def _parse(source, state):
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	386 # parse a simple pattern
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	387 subpattern = SubPattern(state)
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	388
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	389 # precompute constants into local variables
a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	390 subpatternappend = subpattern.append
a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	391 sourceget = source.get
a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	392 sourcematch = source.match
a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	393 _len = len
34635 5f26298811b7 Complete the previous effort to factor out constant expressions Raymond Hettinger <python@rcn.com> parents: 33400 diff changeset	394 PATTERNENDERS = _PATTERNENDERS
5f26298811b7 Complete the previous effort to factor out constant expressions Raymond Hettinger <python@rcn.com> parents: 33400 diff changeset	395 ASSERTCHARS = _ASSERTCHARS
5f26298811b7 Complete the previous effort to factor out constant expressions Raymond Hettinger <python@rcn.com> parents: 33400 diff changeset	396 LOOKBEHINDASSERTCHARS = _LOOKBEHINDASSERTCHARS
5f26298811b7 Complete the previous effort to factor out constant expressions Raymond Hettinger <python@rcn.com> parents: 33400 diff changeset	397 REPEATCODES = _REPEATCODES
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	398
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	399 while 1:
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	400
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	401 if source.next in PATTERNENDERS:
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	402 break # end of subpattern
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	403 this = sourceget()
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	404 if this is None:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	405 break # end of pattern
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	406
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	407 if state.flags & SRE_FLAG_VERBOSE:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	408 # skip whitespace and comments
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	409 if this in WHITESPACE:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	410 continue
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	411 if this == "#":
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	412 while 1:
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	413 this = sourceget()
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	414 if this in (None, "\n"):
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	415 break
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	416 continue
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	417
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	418 if this and this[0] not in SPECIAL_CHARS:
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	419 subpatternappend((LITERAL, ord(this)))
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	420
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	421 elif this == "[":
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	422 # character set
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	423 set = []
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	424 setappend = set.append
a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	425 ## if sourcematch(":"):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	426 ## pass # handle character classes
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	427 if sourcematch("^"):
a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	428 setappend((NEGATE, None))
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	429 # check remaining characters
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	430 start = set[:]
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	431 while 1:
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	432 this = sourceget()
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	433 if this == "]" and set != start:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	434 break
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	435 elif this and this[0] == "\\":
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	436 code1 = _class_escape(source, this)
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	437 elif this:
13201 a093fad92ccf the mad patcher strikes again: Fredrik Lundh <fredrik@pythonware.com> parents: 13198 diff changeset	438 code1 = LITERAL, ord(this)
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	439 else:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	440 raise error, "unexpected end of regular expression"
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	441 if sourcematch("-"):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	442 # potential range
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	443 this = sourceget()
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	444 if this == "]":
15083 02ad9cb52222 SRE didn't handle character category followed by hyphen inside a Fredrik Lundh <fredrik@pythonware.com> parents: 14999 diff changeset	445 if code1[0] is IN:
02ad9cb52222 SRE didn't handle character category followed by hyphen inside a Fredrik Lundh <fredrik@pythonware.com> parents: 14999 diff changeset	446 code1 = code1[1][0]
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	447 setappend(code1)
a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	448 setappend((LITERAL, ord("-")))
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	449 break
28603 7fe33c1541df SF patch #720991 by Gary Herron: Guido van Rossum <guido@python.org> parents: 23627 diff changeset	450 elif this:
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	451 if this[0] == "\\":
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	452 code2 = _class_escape(source, this)
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	453 else:
13201 a093fad92ccf the mad patcher strikes again: Fredrik Lundh <fredrik@pythonware.com> parents: 13198 diff changeset	454 code2 = LITERAL, ord(this)
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	455 if code1[0] != LITERAL or code2[0] != LITERAL:
15858 81e3d8626adf SRE: stricter pattern syntax checking (covers parts of bug #115900) Fredrik Lundh <fredrik@pythonware.com> parents: 15855 diff changeset	456 raise error, "bad character range"
14316 511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	457 lo = code1[1]
511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	458 hi = code2[1]
511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	459 if hi < lo:
15858 81e3d8626adf SRE: stricter pattern syntax checking (covers parts of bug #115900) Fredrik Lundh <fredrik@pythonware.com> parents: 15855 diff changeset	460 raise error, "bad character range"
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	461 setappend((RANGE, (lo, hi)))
28603 7fe33c1541df SF patch #720991 by Gary Herron: Guido van Rossum <guido@python.org> parents: 23627 diff changeset	462 else:
7fe33c1541df SF patch #720991 by Gary Herron: Guido van Rossum <guido@python.org> parents: 23627 diff changeset	463 raise error, "unexpected end of regular expression"
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	464 else:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	465 if code1[0] is IN:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	466 code1 = code1[1][0]
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	467 setappend(code1)
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	468
15855 152d00718006 SRE fixes for 2.1 alpha: Fredrik Lundh <fredrik@pythonware.com> parents: 15355 diff changeset	469 # XXX: <fl> should move set optimization to compiler!
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	470 if _len(set)==1 and set[0][0] is LITERAL:
a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	471 subpatternappend(set[0]) # optimization
a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	472 elif _len(set)==2 and set[0][0] is NEGATE and set[1][0] is LITERAL:
a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	473 subpatternappend((NOT_LITERAL, set[1][1])) # optimization
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	474 else:
15855 152d00718006 SRE fixes for 2.1 alpha: Fredrik Lundh <fredrik@pythonware.com> parents: 15355 diff changeset	475 # XXX: <fl> should add charmap optimization here
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	476 subpatternappend((IN, set))
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	477
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	478 elif this and this[0] in REPEAT_CHARS:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	479 # repeat previous item
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	480 if this == "?":
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	481 min, max = 0, 1
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	482 elif this == "*":
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	483 min, max = 0, MAXREPEAT
16665 de0e8752d5b7 detect attempts to repeat anchors (fixes bug #130748) Fredrik Lundh <fredrik@pythonware.com> parents: 16663 diff changeset	484
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	485 elif this == "+":
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	486 min, max = 1, MAXREPEAT
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	487 elif this == "{":
35544 acfc0e22cd8f Bug #1202493: Fixing SRE parser to handle '{}' as perl does, rather than Gustavo Niemeyer <gustavo@niemeyer.net> parents: 34635 diff changeset	488 if source.next == "}":
acfc0e22cd8f Bug #1202493: Fixing SRE parser to handle '{}' as perl does, rather than Gustavo Niemeyer <gustavo@niemeyer.net> parents: 34635 diff changeset	489 subpatternappend((LITERAL, ord(this)))
acfc0e22cd8f Bug #1202493: Fixing SRE parser to handle '{}' as perl does, rather than Gustavo Niemeyer <gustavo@niemeyer.net> parents: 34635 diff changeset	490 continue
13305 33ce63b06359 - fixed "{ in any other context" bug Fredrik Lundh <fredrik@pythonware.com> parents: 13299 diff changeset	491 here = source.tell()
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	492 min, max = 0, MAXREPEAT
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	493 lo = hi = ""
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	494 while source.next in DIGITS:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	495 lo = lo + source.get()
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	496 if sourcematch(","):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	497 while source.next in DIGITS:
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	498 hi = hi + sourceget()
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	499 else:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	500 hi = lo
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	501 if not sourcematch("}"):
a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	502 subpatternappend((LITERAL, ord(this)))
13305 33ce63b06359 - fixed "{ in any other context" bug Fredrik Lundh <fredrik@pythonware.com> parents: 13299 diff changeset	503 source.seek(here)
33ce63b06359 - fixed "{ in any other context" bug Fredrik Lundh <fredrik@pythonware.com> parents: 13299 diff changeset	504 continue
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	505 if lo:
33257 11a4fbc88a80 PEP 292 classes Template and SafeTemplate are added to the string module. Barry Warsaw <barry@python.org> parents: 31829 diff changeset	506 min = int(lo)
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	507 if hi:
33257 11a4fbc88a80 PEP 292 classes Template and SafeTemplate are added to the string module. Barry Warsaw <barry@python.org> parents: 31829 diff changeset	508 max = int(hi)
15858 81e3d8626adf SRE: stricter pattern syntax checking (covers parts of bug #115900) Fredrik Lundh <fredrik@pythonware.com> parents: 15855 diff changeset	509 if max < min:
81e3d8626adf SRE: stricter pattern syntax checking (covers parts of bug #115900) Fredrik Lundh <fredrik@pythonware.com> parents: 15855 diff changeset	510 raise error, "bad repeat interval"
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	511 else:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	512 raise error, "not supported"
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	513 # figure out which item to repeat
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	514 if subpattern:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	515 item = subpattern[-1:]
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	516 else:
16665 de0e8752d5b7 detect attempts to repeat anchors (fixes bug #130748) Fredrik Lundh <fredrik@pythonware.com> parents: 16663 diff changeset	517 item = None
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	518 if not item or (_len(item) == 1 and item[0][0] == AT):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	519 raise error, "nothing to repeat"
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	520 if item[0][0] in REPEATCODES:
15858 81e3d8626adf SRE: stricter pattern syntax checking (covers parts of bug #115900) Fredrik Lundh <fredrik@pythonware.com> parents: 15855 diff changeset	521 raise error, "multiple repeat"
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	522 if sourcematch("?"):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	523 subpattern[-1] = (MIN_REPEAT, (min, max, item))
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	524 else:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	525 subpattern[-1] = (MAX_REPEAT, (min, max, item))
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	526
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	527 elif this == ".":
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	528 subpatternappend((ANY, None))
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	529
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	530 elif this == "(":
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	531 group = 1
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	532 name = None
30703 8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	533 condgroup = None
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	534 if sourcematch("?"):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	535 group = 0
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	536 # options
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	537 if sourcematch("P"):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	538 # python extensions
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	539 if sourcematch("<"):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	540 # named group: skip forward to end of name
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	541 name = ""
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	542 while 1:
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	543 char = sourceget()
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	544 if char is None:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	545 raise error, "unterminated name"
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	546 if char == ">":
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	547 break
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	548 name = name + char
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	549 group = 1
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	550 if not isname(name):
15858 81e3d8626adf SRE: stricter pattern syntax checking (covers parts of bug #115900) Fredrik Lundh <fredrik@pythonware.com> parents: 15855 diff changeset	551 raise error, "bad character in group name"
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	552 elif sourcematch("="):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	553 # named backreference
13188 92ba928c09b8 - added support for (?P=name) Fredrik Lundh <fredrik@pythonware.com> parents: 13186 diff changeset	554 name = ""
92ba928c09b8 - added support for (?P=name) Fredrik Lundh <fredrik@pythonware.com> parents: 13186 diff changeset	555 while 1:
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	556 char = sourceget()
13188 92ba928c09b8 - added support for (?P=name) Fredrik Lundh <fredrik@pythonware.com> parents: 13186 diff changeset	557 if char is None:
92ba928c09b8 - added support for (?P=name) Fredrik Lundh <fredrik@pythonware.com> parents: 13186 diff changeset	558 raise error, "unterminated name"
92ba928c09b8 - added support for (?P=name) Fredrik Lundh <fredrik@pythonware.com> parents: 13186 diff changeset	559 if char == ")":
92ba928c09b8 - added support for (?P=name) Fredrik Lundh <fredrik@pythonware.com> parents: 13186 diff changeset	560 break
92ba928c09b8 - added support for (?P=name) Fredrik Lundh <fredrik@pythonware.com> parents: 13186 diff changeset	561 name = name + char
92ba928c09b8 - added support for (?P=name) Fredrik Lundh <fredrik@pythonware.com> parents: 13186 diff changeset	562 if not isname(name):
15858 81e3d8626adf SRE: stricter pattern syntax checking (covers parts of bug #115900) Fredrik Lundh <fredrik@pythonware.com> parents: 15855 diff changeset	563 raise error, "bad character in group name"
13188 92ba928c09b8 - added support for (?P=name) Fredrik Lundh <fredrik@pythonware.com> parents: 13186 diff changeset	564 gid = state.groupdict.get(name)
92ba928c09b8 - added support for (?P=name) Fredrik Lundh <fredrik@pythonware.com> parents: 13186 diff changeset	565 if gid is None:
92ba928c09b8 - added support for (?P=name) Fredrik Lundh <fredrik@pythonware.com> parents: 13186 diff changeset	566 raise error, "unknown group name"
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	567 subpatternappend((GROUPREF, gid))
13308 fc2ad6fa08ec - actually enabled charset anchors in the engine (still not Fredrik Lundh <fredrik@pythonware.com> parents: 13306 diff changeset	568 continue
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	569 else:
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	570 char = sourceget()
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	571 if char is None:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	572 raise error, "unexpected end of pattern"
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	573 raise error, "unknown specifier: ?P%s" % char
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	574 elif sourcematch(":"):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	575 # non-capturing group
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	576 group = 2
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	577 elif sourcematch("#"):
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	578 # comment
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	579 while 1:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	580 if source.next is None or source.next == ")":
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	581 break
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	582 sourceget()
a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	583 if not sourcematch(")"):
14316 511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	584 raise error, "unbalanced parenthesis"
511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	585 continue
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	586 elif source.next in ASSERTCHARS:
13198 4deeed58bf76 - fixed lookahead assertions (#10, #11, #12) Fredrik Lundh <fredrik@pythonware.com> parents: 13188 diff changeset	587 # lookahead assertions
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	588 char = sourceget()
13327 c155bccb7a22 - added lookbehind support (?<=pattern), (?<!pattern). Fredrik Lundh <fredrik@pythonware.com> parents: 13310 diff changeset	589 dir = 1
c155bccb7a22 - added lookbehind support (?<=pattern), (?<!pattern). Fredrik Lundh <fredrik@pythonware.com> parents: 13310 diff changeset	590 if char == "<":
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	591 if source.next not in LOOKBEHINDASSERTCHARS:
13327 c155bccb7a22 - added lookbehind support (?<=pattern), (?<!pattern). Fredrik Lundh <fredrik@pythonware.com> parents: 13310 diff changeset	592 raise error, "syntax error"
c155bccb7a22 - added lookbehind support (?<=pattern), (?<!pattern). Fredrik Lundh <fredrik@pythonware.com> parents: 13310 diff changeset	593 dir = -1 # lookbehind
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	594 char = sourceget()
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	595 p = _parse_sub(source, state)
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	596 if not sourcematch(")"):
14316 511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	597 raise error, "unbalanced parenthesis"
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	598 if char == "=":
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	599 subpatternappend((ASSERT, (dir, p)))
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	600 else:
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	601 subpatternappend((ASSERT_NOT, (dir, p)))
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	602 continue
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	603 elif sourcematch("("):
30703 8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	604 # conditional backreference group
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	605 condname = ""
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	606 while 1:
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	607 char = sourceget()
30703 8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	608 if char is None:
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	609 raise error, "unterminated name"
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	610 if char == ")":
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	611 break
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	612 condname = condname + char
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	613 group = 2
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	614 if isname(condname):
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	615 condgroup = state.groupdict.get(condname)
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	616 if condgroup is None:
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	617 raise error, "unknown group name"
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	618 else:
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	619 try:
33257 11a4fbc88a80 PEP 292 classes Template and SafeTemplate are added to the string module. Barry Warsaw <barry@python.org> parents: 31829 diff changeset	620 condgroup = int(condname)
30703 8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	621 except ValueError:
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	622 raise error, "bad character in group name"
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	623 else:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	624 # flags
23620 a1de0bab5475 SF 563203. Replaced 'has_key()' with 'in'. Raymond Hettinger <python@rcn.com> parents: 22867 diff changeset	625 if not source.next in FLAGS:
15858 81e3d8626adf SRE: stricter pattern syntax checking (covers parts of bug #115900) Fredrik Lundh <fredrik@pythonware.com> parents: 15855 diff changeset	626 raise error, "unexpected end of pattern"
23620 a1de0bab5475 SF 563203. Replaced 'has_key()' with 'in'. Raymond Hettinger <python@rcn.com> parents: 22867 diff changeset	627 while source.next in FLAGS:
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	628 state.flags = state.flags \| FLAGS[sourceget()]
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	629 if group:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	630 # parse group contents
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	631 if group == 2:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	632 # anonymous group
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	633 group = None
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	634 else:
15355 2a3696d32721 -- properly reset groups in findall (bug #117612) Fredrik Lundh <fredrik@pythonware.com> parents: 15091 diff changeset	635 group = state.opengroup(name)
30703 8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	636 if condgroup:
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	637 p = _parse_sub_cond(source, state, condgroup)
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	638 else:
8282aa1f9343 Implemented non-recursive SRE matching. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 28689 diff changeset	639 p = _parse_sub(source, state)
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	640 if not sourcematch(")"):
14316 511123dd8424 closes bug #112468 (and all the other bugs that surfaced when Fredrik Lundh <fredrik@pythonware.com> parents: 13886 diff changeset	641 raise error, "unbalanced parenthesis"
15355 2a3696d32721 -- properly reset groups in findall (bug #117612) Fredrik Lundh <fredrik@pythonware.com> parents: 15091 diff changeset	642 if group is not None:
2a3696d32721 -- properly reset groups in findall (bug #117612) Fredrik Lundh <fredrik@pythonware.com> parents: 15091 diff changeset	643 state.closegroup(group)
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	644 subpatternappend((SUBPATTERN, (group, p)))
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	645 else:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	646 while 1:
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	647 char = sourceget()
15858 81e3d8626adf SRE: stricter pattern syntax checking (covers parts of bug #115900) Fredrik Lundh <fredrik@pythonware.com> parents: 15855 diff changeset	648 if char is None:
81e3d8626adf SRE: stricter pattern syntax checking (covers parts of bug #115900) Fredrik Lundh <fredrik@pythonware.com> parents: 15855 diff changeset	649 raise error, "unexpected end of pattern"
81e3d8626adf SRE: stricter pattern syntax checking (covers parts of bug #115900) Fredrik Lundh <fredrik@pythonware.com> parents: 15855 diff changeset	650 if char == ")":
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	651 break
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	652 raise error, "unknown extension"
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	653
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	654 elif this == "^":
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	655 subpatternappend((AT, AT_BEGINNING))
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	656
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	657 elif this == "$":
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	658 subpattern.append((AT, AT_END))
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	659
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	660 elif this and this[0] == "\\":
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	661 code = _escape(source, this, state)
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	662 subpatternappend(code)
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	663
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	664 else:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	665 raise error, "parser error"
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	666
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	667 return subpattern
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	668
13886 62494a96dcca -- reset marks if repeat_one tail doesn't match Fredrik Lundh <fredrik@pythonware.com> parents: 13795 diff changeset	669 def parse(str, flags=0, pattern=None):
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	670 # parse 're' pattern into list of (opcode, argument) tuples
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	671
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	672 source = Tokenizer(str)
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	673
13886 62494a96dcca -- reset marks if repeat_one tail doesn't match Fredrik Lundh <fredrik@pythonware.com> parents: 13795 diff changeset	674 if pattern is None:
62494a96dcca -- reset marks if repeat_one tail doesn't match Fredrik Lundh <fredrik@pythonware.com> parents: 13795 diff changeset	675 pattern = Pattern()
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	676 pattern.flags = flags
15858 81e3d8626adf SRE: stricter pattern syntax checking (covers parts of bug #115900) Fredrik Lundh <fredrik@pythonware.com> parents: 15855 diff changeset	677 pattern.str = str
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	678
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	679 p = _parse_sub(source, pattern, 0)
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	680
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	681 tail = source.get()
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	682 if tail == ")":
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	683 raise error, "unbalanced parenthesis"
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	684 elif tail:
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	685 raise error, "bogus characters at end of regular expression"
edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	686
15855 152d00718006 SRE fixes for 2.1 alpha: Fredrik Lundh <fredrik@pythonware.com> parents: 15355 diff changeset	687 if flags & SRE_FLAG_DEBUG:
152d00718006 SRE fixes for 2.1 alpha: Fredrik Lundh <fredrik@pythonware.com> parents: 15355 diff changeset	688 p.dump()
13677 edb1a8e6b859 -- SRE 0.9.6 sync. this includes: Fredrik Lundh <fredrik@pythonware.com> parents: 13328 diff changeset	689
14999 1a3b5c8d6730 Recompile pattern if (?x) flag was found inside the pattern during the Fredrik Lundh <fredrik@pythonware.com> parents: 14797 diff changeset	690 if not (flags & SRE_FLAG_VERBOSE) and p.pattern.flags & SRE_FLAG_VERBOSE:
1a3b5c8d6730 Recompile pattern if (?x) flag was found inside the pattern during the Fredrik Lundh <fredrik@pythonware.com> parents: 14797 diff changeset	691 # the VERBOSE flag was switched on inside the pattern. to be
1a3b5c8d6730 Recompile pattern if (?x) flag was found inside the pattern during the Fredrik Lundh <fredrik@pythonware.com> parents: 14797 diff changeset	692 # on the safe side, we'll parse the whole thing again...
1a3b5c8d6730 Recompile pattern if (?x) flag was found inside the pattern during the Fredrik Lundh <fredrik@pythonware.com> parents: 14797 diff changeset	693 return parse(str, p.pattern.flags)
1a3b5c8d6730 Recompile pattern if (?x) flag was found inside the pattern during the Fredrik Lundh <fredrik@pythonware.com> parents: 14797 diff changeset	694
12175 0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	695 return p
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast. Guido van Rossum <guido@python.org> parents: diff changeset	696
13118 7ff40da08c2a towards 1.6b1 Fredrik Lundh <fredrik@pythonware.com> parents: 12903 diff changeset	697 def parse_template(source, pattern):
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	698 # parse 're' replacement string into list of literals and
9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	699 # group references
9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	700 s = Tokenizer(source)
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	701 sget = s.get
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	702 p = []
9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	703 a = p.append
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	704 def literal(literal, p=p, pappend=a):
17144 bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	705 if p and p[-1][0] is LITERAL:
bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	706 p[-1] = LITERAL, p[-1][1] + literal
bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	707 else:
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	708 pappend((LITERAL, literal))
17144 bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	709 sep = source[:0]
bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	710 if type(sep) is type(""):
19971 d4c965316610 fixed #449964: sre.sub raises an exception if the template contains a Fredrik Lundh <fredrik@pythonware.com> parents: 19693 diff changeset	711 makechar = chr
17144 bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	712 else:
19971 d4c965316610 fixed #449964: sre.sub raises an exception if the template contains a Fredrik Lundh <fredrik@pythonware.com> parents: 19693 diff changeset	713 makechar = unichr
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	714 while 1:
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	715 this = sget()
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	716 if this is None:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	717 break # end of replacement string
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	718 if this and this[0] == "\\":
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	719 # group
33397 98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	720 c = this[1:2]
98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	721 if c == "g":
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	722 name = ""
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	723 if s.match("<"):
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	724 while 1:
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	725 char = sget()
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	726 if char is None:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	727 raise error, "unterminated group name"
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	728 if char == ">":
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	729 break
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	730 name = name + char
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	731 if not name:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	732 raise error, "bad group name"
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	733 try:
33257 11a4fbc88a80 PEP 292 classes Template and SafeTemplate are added to the string module. Barry Warsaw <barry@python.org> parents: 31829 diff changeset	734 index = int(name)
33397 98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	735 if index < 0:
98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	736 raise error, "negative group number"
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	737 except ValueError:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	738 if not isname(name):
15858 81e3d8626adf SRE: stricter pattern syntax checking (covers parts of bug #115900) Fredrik Lundh <fredrik@pythonware.com> parents: 15855 diff changeset	739 raise error, "bad character in group name"
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	740 try:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	741 index = pattern.groupindex[name]
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	742 except KeyError:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	743 raise IndexError, "unknown group name"
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	744 a((MARK, index))
33397 98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	745 elif c == "0":
98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	746 if s.next in OCTDIGITS:
98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	747 this = this + sget()
98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	748 if s.next in OCTDIGITS:
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	749 this = this + sget()
33397 98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	750 literal(makechar(int(this[1:], 8) & 0xff))
98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	751 elif c in DIGITS:
98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	752 isoctal = False
98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	753 if s.next in DIGITS:
98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	754 this = this + sget()
33400 b9945335da27 Minor aesthetic change in sre_parse.py, as suggested by Mike Coleman. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33397 diff changeset	755 if (c in OCTDIGITS and this[2] in OCTDIGITS and
b9945335da27 Minor aesthetic change in sre_parse.py, as suggested by Mike Coleman. Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33397 diff changeset	756 s.next in OCTDIGITS):
33397 98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	757 this = this + sget()
98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	758 isoctal = True
98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	759 literal(makechar(int(this[1:], 8) & 0xff))
98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	760 if not isoctal:
98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	761 a((MARK, int(this[1:])))
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	762 else:
c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	763 try:
19971 d4c965316610 fixed #449964: sre.sub raises an exception if the template contains a Fredrik Lundh <fredrik@pythonware.com> parents: 19693 diff changeset	764 this = makechar(ESCAPES[this][1])
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	765 except KeyError:
17144 bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	766 pass
bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	767 literal(this)
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	768 else:
17144 bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	769 literal(this)
bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	770 # convert template to groups and literals lists
bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	771 i = 0
bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	772 groups = []
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	773 groupsappend = groups.append
a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	774 literals = [None] * len(p)
17144 bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	775 for c, s in p:
bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	776 if c is MARK:
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	777 groupsappend((i, s))
a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	778 # literal[i] is already None
17144 bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	779 else:
31829 a76ad83a413f Simple Optimizations: Raymond Hettinger <python@rcn.com> parents: 31389 diff changeset	780 literals[i] = s
17144 bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	781 i = i + 1
bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	782 return groups, literals
12903 9ee1e594571d Patch from /F: this patch brings the CVS version of SRE in sync with the Andrew M. Kuchling <amk@amk.ca> parents: 12339 diff changeset	783
13118 7ff40da08c2a towards 1.6b1 Fredrik Lundh <fredrik@pythonware.com> parents: 12903 diff changeset	784 def expand_template(template, match):
17144 bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	785 g = match.group
13201 a093fad92ccf the mad patcher strikes again: Fredrik Lundh <fredrik@pythonware.com> parents: 13198 diff changeset	786 sep = match.string[:0]
17144 bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	787 groups, literals = template
bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	788 literals = literals[:]
bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	789 try:
bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	790 for index, group in groups:
bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	791 literals[index] = s = g(group)
13186 c45bc258f833 - pedantic: make sure "python -t" doesn't complain... Fredrik Lundh <fredrik@pythonware.com> parents: 13166 diff changeset	792 if s is None:
33397 98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	793 raise error, "unmatched group"
17144 bbf5ea2a22e5 sre 2.1b2 update: Fredrik Lundh <fredrik@pythonware.com> parents: 16665 diff changeset	794 except IndexError:
33397 98c8017e8535 Applying modified version of patch #1018386, which fixes Gustavo Niemeyer <gustavo@niemeyer.net> parents: 33257 diff changeset	795 raise error, "invalid group reference"
33257 11a4fbc88a80 PEP 292 classes Template and SafeTemplate are added to the string module. Barry Warsaw <barry@python.org> parents: 31829 diff changeset	796 return sep.join(literals)

Mercurial > cpython-lru_cache-weakref

annotate Lib/sre_parse.py @ 58925:eecf1bef052d 2.6