Mercurial > cpython-lru_cache-weakref

annotate Lib/re.py @ 80661:94de77bd0b4b 2.7

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Issue #16194: document imp.load_dynamic problems when read several extension modules from the same binary object file.
author Andrew Svetlov <andrew.svetlov@gmail.com>
date Thu, 29 Nov 2012 19:51:28 +0200
parents 32c8af00eef2
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
12175
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast.
Guido van Rossum <guido@python.org>
parents:
diff changeset
1 #
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast.
Guido van Rossum <guido@python.org>
parents:
diff changeset
2 # Secret Labs' Regular Expression Engine
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast.
Guido van Rossum <guido@python.org>
parents:
diff changeset
3 #
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast.
Guido van Rossum <guido@python.org>
parents:
diff changeset
4 # re-compatible interface for the sre matching engine
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast.
Guido van Rossum <guido@python.org>
parents:
diff changeset
5 #
15855
152d00718006 SRE fixes for 2.1 alpha:
Fredrik Lundh <fredrik@pythonware.com>
parents: 14726
diff changeset
6 # Copyright (c) 1998-2001 by Secret Labs AB. All rights reserved.
12175
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast.
Guido van Rossum <guido@python.org>
parents:
diff changeset
7 #
13789
4edd13eb759d SRE 0.9.8: passes the entire test suite
Fredrik Lundh <fredrik@pythonware.com>
parents: 13677
diff changeset
8 # This version of the SRE library can be redistributed under CNRI's
4edd13eb759d SRE 0.9.8: passes the entire test suite
Fredrik Lundh <fredrik@pythonware.com>
parents: 13677
diff changeset
9 # Python 1.6 license. For any other use, please contact Secret Labs
4edd13eb759d SRE 0.9.8: passes the entire test suite
Fredrik Lundh <fredrik@pythonware.com>
parents: 13677
diff changeset
10 # AB (info@pythonware.com).
4edd13eb759d SRE 0.9.8: passes the entire test suite
Fredrik Lundh <fredrik@pythonware.com>
parents: 13677
diff changeset
11 #
12175
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast.
Guido van Rossum <guido@python.org>
parents:
diff changeset
12 # Portions of this engine have been developed in cooperation with
13789
4edd13eb759d SRE 0.9.8: passes the entire test suite
Fredrik Lundh <fredrik@pythonware.com>
parents: 13677
diff changeset
13 # CNRI. Hewlett-Packard provided funding for 1.6 integration and
12175
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast.
Guido van Rossum <guido@python.org>
parents:
diff changeset
14 # other compatibility work.
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast.
Guido van Rossum <guido@python.org>
parents:
diff changeset
15 #
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast.
Guido van Rossum <guido@python.org>
parents:
diff changeset
16
19695
9830d00b887d Convert docstring to "raw" string.
Fred Drake <fdrake@acm.org>
parents: 19693
diff changeset
17 r"""Support for regular expressions (RE).
19693
617f5e72b0ec Added docstrings by Neal Norwitz. This closes SF bug #450980.
Fred Drake <fdrake@acm.org>
parents: 19130
diff changeset
18
617f5e72b0ec Added docstrings by Neal Norwitz. This closes SF bug #450980.
Fred Drake <fdrake@acm.org>
parents: 19130
diff changeset
19 This module provides regular expression matching operations similar to
20592
82bcfac89642 fixed character set description in docstring (SRE uses Python
Fredrik Lundh <fredrik@pythonware.com>
parents: 20590
diff changeset
20 those found in Perl. It supports both 8-bit and Unicode strings; both
82bcfac89642 fixed character set description in docstring (SRE uses Python
Fredrik Lundh <fredrik@pythonware.com>
parents: 20590
diff changeset
21 the pattern and the strings being processed can contain null bytes and
82bcfac89642 fixed character set description in docstring (SRE uses Python
Fredrik Lundh <fredrik@pythonware.com>
parents: 20590
diff changeset
22 characters outside the US ASCII range.
19693
617f5e72b0ec Added docstrings by Neal Norwitz. This closes SF bug #450980.
Fred Drake <fdrake@acm.org>
parents: 19130
diff changeset
23
20592
82bcfac89642 fixed character set description in docstring (SRE uses Python
Fredrik Lundh <fredrik@pythonware.com>
parents: 20590
diff changeset
24 Regular expressions can contain both special and ordinary characters.
82bcfac89642 fixed character set description in docstring (SRE uses Python
Fredrik Lundh <fredrik@pythonware.com>
parents: 20590
diff changeset
25 Most ordinary characters, like "A", "a", or "0", are the simplest
82bcfac89642 fixed character set description in docstring (SRE uses Python
Fredrik Lundh <fredrik@pythonware.com>
parents: 20590
diff changeset
26 regular expressions; they simply match themselves. You can
19693
617f5e72b0ec Added docstrings by Neal Norwitz. This closes SF bug #450980.
Fred Drake <fdrake@acm.org>
parents: 19130
diff changeset
27 concatenate ordinary characters, so last matches the string 'last'.
617f5e72b0ec Added docstrings by Neal Norwitz. This closes SF bug #450980.
Fred Drake <fdrake@acm.org>
parents: 19130
diff changeset
28
617f5e72b0ec Added docstrings by Neal Norwitz. This closes SF bug #450980.
Fred Drake <fdrake@acm.org>
parents: 19130
diff changeset
29 The special characters are:
617f5e72b0ec Added docstrings by Neal Norwitz. This closes SF bug #450980.
Fred Drake <fdrake@acm.org>
parents: 19130
diff changeset
30 "." Matches any character except a newline.
617f5e72b0ec Added docstrings by Neal Norwitz. This closes SF bug #450980.
Fred Drake <fdrake@acm.org>
parents: 19130
diff changeset
31 "^" Matches the start of the string.
45171
dac4217e9965 Backport doc fixes from 2.6. These are all things that were already
Guido van Rossum <guido@python.org>
parents: 45170
diff changeset
32 "$" Matches the end of the string or just before the newline at
dac4217e9965 Backport doc fixes from 2.6. These are all things that were already
Guido van Rossum <guido@python.org>
parents: 45170
diff changeset
33 the end of the string.
19693
617f5e72b0ec Added docstrings by Neal Norwitz. This closes SF bug #450980.
Fred Drake <fdrake@acm.org>
parents: 19130
diff changeset
34 "*" Matches 0 or more (greedy) repetitions of the preceding RE.
617f5e72b0ec Added docstrings by Neal Norwitz. This closes SF bug #450980.
Fred Drake <fdrake@acm.org>
parents: 19130
diff changeset
35 Greedy means that it will match as many repetitions as possible.
617f5e72b0ec Added docstrings by Neal Norwitz. This closes SF bug #450980.
Fred Drake <fdrake@acm.org>
parents: 19130
diff changeset
36 "+" Matches 1 or more (greedy) repetitions of the preceding RE.
617f5e72b0ec Added docstrings by Neal Norwitz. This closes SF bug #450980.
Fred Drake <fdrake@acm.org>
parents: 19130
diff changeset
37 "?" Matches 0 or 1 (greedy) of the preceding RE.
617f5e72b0ec Added docstrings by Neal Norwitz. This closes SF bug #450980.
Fred Drake <fdrake@acm.org>
parents: 19130
diff changeset
38 *?,+?,?? Non-greedy versions of the previous three special characters.
617f5e72b0ec Added docstrings by Neal Norwitz. This closes SF bug #450980.
Fred Drake <fdrake@acm.org>
parents: 19130
diff changeset
39 {m,n} Matches from m to n repetitions of the preceding RE.
617f5e72b0ec Added docstrings by Neal Norwitz. This closes SF bug #450980.
Fred Drake <fdrake@acm.org>
parents: 19130
diff changeset
40 {m,n}? Non-greedy version of the above.
45171
dac4217e9965 Backport doc fixes from 2.6. These are all things that were already
Guido van Rossum <guido@python.org>
parents: 45170
diff changeset
41 "\\" Either escapes special characters or signals a special sequence.
19693
617f5e72b0ec Added docstrings by Neal Norwitz. This closes SF bug #450980.
Fred Drake <fdrake@acm.org>
parents: 19130
diff changeset
42 [] Indicates a set of characters.
617f5e72b0ec Added docstrings by Neal Norwitz. This closes SF bug #450980.
Fred Drake <fdrake@acm.org>
parents: 19130
diff changeset
43 A "^" as the first character indicates a complementing set.
617f5e72b0ec Added docstrings by Neal Norwitz. This closes SF bug #450980.
Fred Drake <fdrake@acm.org>
parents: 19130
diff changeset
44 "|" A|B, creates an RE that will match either A or B.
617f5e72b0ec Added docstrings by Neal Norwitz. This closes SF bug #450980.
Fred Drake <fdrake@acm.org>
parents: 19130
diff changeset
45 (...) Matches the RE inside the parentheses.
617f5e72b0ec Added docstrings by Neal Norwitz. This closes SF bug #450980.
Fred Drake <fdrake@acm.org>
parents: 19130
diff changeset
46 The contents can be retrieved or matched later in the string.
20592
82bcfac89642 fixed character set description in docstring (SRE uses Python
Fredrik Lundh <fredrik@pythonware.com>
parents: 20590
diff changeset
47 (?iLmsux) Set the I, L, M, S, U, or X flag for the RE (see below).
19693
617f5e72b0ec Added docstrings by Neal Norwitz. This closes SF bug #450980.
Fred Drake <fdrake@acm.org>
parents: 19130
diff changeset
48 (?:...) Non-grouping version of regular parentheses.
617f5e72b0ec Added docstrings by Neal Norwitz. This closes SF bug #450980.
Fred Drake <fdrake@acm.org>
parents: 19130
diff changeset
49 (?P<name>...) The substring matched by the group is accessible by name.
617f5e72b0ec Added docstrings by Neal Norwitz. This closes SF bug #450980.
Fred Drake <fdrake@acm.org>
parents: 19130
diff changeset
50 (?P=name) Matches the text matched earlier by the group named name.
617f5e72b0ec Added docstrings by Neal Norwitz. This closes SF bug #450980.
Fred Drake <fdrake@acm.org>
parents: 19130
diff changeset
51 (?#...) A comment; ignored.
617f5e72b0ec Added docstrings by Neal Norwitz. This closes SF bug #450980.
Fred Drake <fdrake@acm.org>
parents: 19130
diff changeset
52 (?=...) Matches if ... matches next, but doesn't consume the string.
617f5e72b0ec Added docstrings by Neal Norwitz. This closes SF bug #450980.
Fred Drake <fdrake@acm.org>
parents: 19130
diff changeset
53 (?!...) Matches if ... doesn't match next.
45171
dac4217e9965 Backport doc fixes from 2.6. These are all things that were already
Guido van Rossum <guido@python.org>
parents: 45170
diff changeset
54 (?<=...) Matches if preceded by ... (must be fixed length).
dac4217e9965 Backport doc fixes from 2.6. These are all things that were already
Guido van Rossum <guido@python.org>
parents: 45170
diff changeset
55 (?<!...) Matches if not preceded by ... (must be fixed length).
dac4217e9965 Backport doc fixes from 2.6. These are all things that were already
Guido van Rossum <guido@python.org>
parents: 45170
diff changeset
56 (?(id/name)yes|no) Matches yes pattern if the group with id/name matched,
dac4217e9965 Backport doc fixes from 2.6. These are all things that were already
Guido van Rossum <guido@python.org>
parents: 45170
diff changeset
57 the (optional) no pattern otherwise.
19693
617f5e72b0ec Added docstrings by Neal Norwitz. This closes SF bug #450980.
Fred Drake <fdrake@acm.org>
parents: 19130
diff changeset
58
617f5e72b0ec Added docstrings by Neal Norwitz. This closes SF bug #450980.
Fred Drake <fdrake@acm.org>
parents: 19130
diff changeset
59 The special sequences consist of "\\" and a character from the list
20592
82bcfac89642 fixed character set description in docstring (SRE uses Python
Fredrik Lundh <fredrik@pythonware.com>
parents: 20590
diff changeset
60 below. If the ordinary character is not on the list, then the
19693
617f5e72b0ec Added docstrings by Neal Norwitz. This closes SF bug #450980.
Fred Drake <fdrake@acm.org>
parents: 19130
diff changeset
61 resulting RE will match the second character.
19695
9830d00b887d Convert docstring to "raw" string.
Fred Drake <fdrake@acm.org>
parents: 19693
diff changeset
62 \number Matches the contents of the group of the same number.
9830d00b887d Convert docstring to "raw" string.
Fred Drake <fdrake@acm.org>
parents: 19693
diff changeset
63 \A Matches only at the start of the string.
9830d00b887d Convert docstring to "raw" string.
Fred Drake <fdrake@acm.org>
parents: 19693
diff changeset
64 \Z Matches only at the end of the string.
9830d00b887d Convert docstring to "raw" string.
Fred Drake <fdrake@acm.org>
parents: 19693
diff changeset
65 \b Matches the empty string, but only at the start or end of a word.
9830d00b887d Convert docstring to "raw" string.
Fred Drake <fdrake@acm.org>
parents: 19693
diff changeset
66 \B Matches the empty string, but not at the start or end of a word.
9830d00b887d Convert docstring to "raw" string.
Fred Drake <fdrake@acm.org>
parents: 19693
diff changeset
67 \d Matches any decimal digit; equivalent to the set [0-9].
9830d00b887d Convert docstring to "raw" string.
Fred Drake <fdrake@acm.org>
parents: 19693
diff changeset
68 \D Matches any non-digit character; equivalent to the set [^0-9].
9830d00b887d Convert docstring to "raw" string.
Fred Drake <fdrake@acm.org>
parents: 19693
diff changeset
69 \s Matches any whitespace character; equivalent to [ \t\n\r\f\v].
9830d00b887d Convert docstring to "raw" string.
Fred Drake <fdrake@acm.org>
parents: 19693
diff changeset
70 \S Matches any non-whitespace character; equiv. to [^ \t\n\r\f\v].
9830d00b887d Convert docstring to "raw" string.
Fred Drake <fdrake@acm.org>
parents: 19693
diff changeset
71 \w Matches any alphanumeric character; equivalent to [a-zA-Z0-9_].
19693
617f5e72b0ec Added docstrings by Neal Norwitz. This closes SF bug #450980.
Fred Drake <fdrake@acm.org>
parents: 19130
diff changeset
72 With LOCALE, it will match the set [0-9_] plus characters defined
617f5e72b0ec Added docstrings by Neal Norwitz. This closes SF bug #450980.
Fred Drake <fdrake@acm.org>
parents: 19130
diff changeset
73 as letters for the current locale.
19695
9830d00b887d Convert docstring to "raw" string.
Fred Drake <fdrake@acm.org>
parents: 19693
diff changeset
74 \W Matches the complement of \w.
9830d00b887d Convert docstring to "raw" string.
Fred Drake <fdrake@acm.org>
parents: 19693
diff changeset
75 \\ Matches a literal backslash.
19693
617f5e72b0ec Added docstrings by Neal Norwitz. This closes SF bug #450980.
Fred Drake <fdrake@acm.org>
parents: 19130
diff changeset
76
617f5e72b0ec Added docstrings by Neal Norwitz. This closes SF bug #450980.
Fred Drake <fdrake@acm.org>
parents: 19130
diff changeset
77 This module exports the following functions:
617f5e72b0ec Added docstrings by Neal Norwitz. This closes SF bug #450980.
Fred Drake <fdrake@acm.org>
parents: 19130
diff changeset
78 match Match a regular expression pattern to the beginning of a string.
617f5e72b0ec Added docstrings by Neal Norwitz. This closes SF bug #450980.
Fred Drake <fdrake@acm.org>
parents: 19130
diff changeset
79 search Search a string for the presence of a pattern.
617f5e72b0ec Added docstrings by Neal Norwitz. This closes SF bug #450980.
Fred Drake <fdrake@acm.org>
parents: 19130
diff changeset
80 sub Substitute occurrences of a pattern found in a string.
617f5e72b0ec Added docstrings by Neal Norwitz. This closes SF bug #450980.
Fred Drake <fdrake@acm.org>
parents: 19130
diff changeset
81 subn Same as sub, but also return the number of substitutions made.
617f5e72b0ec Added docstrings by Neal Norwitz. This closes SF bug #450980.
Fred Drake <fdrake@acm.org>
parents: 19130
diff changeset
82 split Split a string by the occurrences of a pattern.
617f5e72b0ec Added docstrings by Neal Norwitz. This closes SF bug #450980.
Fred Drake <fdrake@acm.org>
parents: 19130
diff changeset
83 findall Find all occurrences of a pattern in a string.
45171
dac4217e9965 Backport doc fixes from 2.6. These are all things that were already
Guido van Rossum <guido@python.org>
parents: 45170
diff changeset
84 finditer Return an iterator yielding a match object for each match.
19693
617f5e72b0ec Added docstrings by Neal Norwitz. This closes SF bug #450980.
Fred Drake <fdrake@acm.org>
parents: 19130
diff changeset
85 compile Compile a pattern into a RegexObject.
617f5e72b0ec Added docstrings by Neal Norwitz. This closes SF bug #450980.
Fred Drake <fdrake@acm.org>
parents: 19130
diff changeset
86 purge Clear the regular expression cache.
617f5e72b0ec Added docstrings by Neal Norwitz. This closes SF bug #450980.
Fred Drake <fdrake@acm.org>
parents: 19130
diff changeset
87 escape Backslash all non-alphanumerics in a string.
617f5e72b0ec Added docstrings by Neal Norwitz. This closes SF bug #450980.
Fred Drake <fdrake@acm.org>
parents: 19130
diff changeset
88
617f5e72b0ec Added docstrings by Neal Norwitz. This closes SF bug #450980.
Fred Drake <fdrake@acm.org>
parents: 19130
diff changeset
89 Some of the functions in this module takes flags as optional parameters:
617f5e72b0ec Added docstrings by Neal Norwitz. This closes SF bug #450980.
Fred Drake <fdrake@acm.org>
parents: 19130
diff changeset
90 I IGNORECASE Perform case-insensitive matching.
617f5e72b0ec Added docstrings by Neal Norwitz. This closes SF bug #450980.
Fred Drake <fdrake@acm.org>
parents: 19130
diff changeset
91 L LOCALE Make \w, \W, \b, \B, dependent on the current locale.
45171
dac4217e9965 Backport doc fixes from 2.6. These are all things that were already
Guido van Rossum <guido@python.org>
parents: 45170
diff changeset
92 M MULTILINE "^" matches the beginning of lines (after a newline)
dac4217e9965 Backport doc fixes from 2.6. These are all things that were already
Guido van Rossum <guido@python.org>
parents: 45170
diff changeset
93 as well as the string.
dac4217e9965 Backport doc fixes from 2.6. These are all things that were already
Guido van Rossum <guido@python.org>
parents: 45170
diff changeset
94 "$" matches the end of lines (before a newline) as well
dac4217e9965 Backport doc fixes from 2.6. These are all things that were already
Guido van Rossum <guido@python.org>
parents: 45170
diff changeset
95 as the end of the string.
19693
617f5e72b0ec Added docstrings by Neal Norwitz. This closes SF bug #450980.
Fred Drake <fdrake@acm.org>
parents: 19130
diff changeset
96 S DOTALL "." matches any character at all, including the newline.
617f5e72b0ec Added docstrings by Neal Norwitz. This closes SF bug #450980.
Fred Drake <fdrake@acm.org>
parents: 19130
diff changeset
97 X VERBOSE Ignore whitespace and comments for nicer looking RE's.
19963
969e306b62fd an SRE bugfix a day keeps Guido away...
Fredrik Lundh <fredrik@pythonware.com>
parents: 19695
diff changeset
98 U UNICODE Make \w, \W, \b, \B, dependent on the Unicode locale.
19693
617f5e72b0ec Added docstrings by Neal Norwitz. This closes SF bug #450980.
Fred Drake <fdrake@acm.org>
parents: 19130
diff changeset
99
617f5e72b0ec Added docstrings by Neal Norwitz. This closes SF bug #450980.
Fred Drake <fdrake@acm.org>
parents: 19130
diff changeset
100 This module also defines an exception 'error'.
617f5e72b0ec Added docstrings by Neal Norwitz. This closes SF bug #450980.
Fred Drake <fdrake@acm.org>
parents: 19130
diff changeset
101
617f5e72b0ec Added docstrings by Neal Norwitz. This closes SF bug #450980.
Fred Drake <fdrake@acm.org>
parents: 19130
diff changeset
102 """
19963
969e306b62fd an SRE bugfix a day keeps Guido away...
Fredrik Lundh <fredrik@pythonware.com>
parents: 19695
diff changeset
103
20660
c05d4895234c (experimental) "finditer" method/function. this works pretty much
Fredrik Lundh <fredrik@pythonware.com>
parents: 20592
diff changeset
104 import sys
12819
a24d9ad6786d Fredrik Lundh: here's the 96.6% version of SRE
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 12548
diff changeset
105 import sre_compile
13118
7ff40da08c2a towards 1.6b1
Fredrik Lundh <fredrik@pythonware.com>
parents: 12955
diff changeset
106 import sre_parse
12175
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast.
Guido van Rossum <guido@python.org>
parents:
diff changeset
107
16663
94a38443f99b - restored 1.5.2 compatibility (sorry, eric)
Fredrik Lundh <fredrik@pythonware.com>
parents: 16613
diff changeset
108 # public symbols
94a38443f99b - restored 1.5.2 compatibility (sorry, eric)
Fredrik Lundh <fredrik@pythonware.com>
parents: 16613
diff changeset
109 __all__ = [ "match", "search", "sub", "subn", "split", "findall",
94a38443f99b - restored 1.5.2 compatibility (sorry, eric)
Fredrik Lundh <fredrik@pythonware.com>
parents: 16613
diff changeset
110 "compile", "purge", "template", "escape", "I", "L", "M", "S", "X",
94a38443f99b - restored 1.5.2 compatibility (sorry, eric)
Fredrik Lundh <fredrik@pythonware.com>
parents: 16613
diff changeset
111 "U", "IGNORECASE", "LOCALE", "MULTILINE", "DOTALL", "VERBOSE",
94a38443f99b - restored 1.5.2 compatibility (sorry, eric)
Fredrik Lundh <fredrik@pythonware.com>
parents: 16613
diff changeset
112 "UNICODE", "error" ]
94a38443f99b - restored 1.5.2 compatibility (sorry, eric)
Fredrik Lundh <fredrik@pythonware.com>
parents: 16613
diff changeset
113
20589
d3298435d85b rewrote the pattern.sub and pattern.subn methods in C
Fredrik Lundh <fredrik@pythonware.com>
parents: 20534
diff changeset
114 __version__ = "2.2.1"
17144
bbf5ea2a22e5 sre 2.1b2 update:
Fredrik Lundh <fredrik@pythonware.com>
parents: 16663
diff changeset
115
12819
a24d9ad6786d Fredrik Lundh: here's the 96.6% version of SRE
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 12548
diff changeset
116 # flags
15855
152d00718006 SRE fixes for 2.1 alpha:
Fredrik Lundh <fredrik@pythonware.com>
parents: 14726
diff changeset
117 I = IGNORECASE = sre_compile.SRE_FLAG_IGNORECASE # ignore case
152d00718006 SRE fixes for 2.1 alpha:
Fredrik Lundh <fredrik@pythonware.com>
parents: 14726
diff changeset
118 L = LOCALE = sre_compile.SRE_FLAG_LOCALE # assume current 8-bit locale
152d00718006 SRE fixes for 2.1 alpha:
Fredrik Lundh <fredrik@pythonware.com>
parents: 14726
diff changeset
119 U = UNICODE = sre_compile.SRE_FLAG_UNICODE # assume unicode locale
152d00718006 SRE fixes for 2.1 alpha:
Fredrik Lundh <fredrik@pythonware.com>
parents: 14726
diff changeset
120 M = MULTILINE = sre_compile.SRE_FLAG_MULTILINE # make anchors look for newline
152d00718006 SRE fixes for 2.1 alpha:
Fredrik Lundh <fredrik@pythonware.com>
parents: 14726
diff changeset
121 S = DOTALL = sre_compile.SRE_FLAG_DOTALL # make dot match newline
152d00718006 SRE fixes for 2.1 alpha:
Fredrik Lundh <fredrik@pythonware.com>
parents: 14726
diff changeset
122 X = VERBOSE = sre_compile.SRE_FLAG_VERBOSE # ignore whitespace and comments
12175
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast.
Guido van Rossum <guido@python.org>
parents:
diff changeset
123
15855
152d00718006 SRE fixes for 2.1 alpha:
Fredrik Lundh <fredrik@pythonware.com>
parents: 14726
diff changeset
124 # sre extensions (experimental, don't rely on these)
152d00718006 SRE fixes for 2.1 alpha:
Fredrik Lundh <fredrik@pythonware.com>
parents: 14726
diff changeset
125 T = TEMPLATE = sre_compile.SRE_FLAG_TEMPLATE # disable backtracking
152d00718006 SRE fixes for 2.1 alpha:
Fredrik Lundh <fredrik@pythonware.com>
parents: 14726
diff changeset
126 DEBUG = sre_compile.SRE_FLAG_DEBUG # dump pattern after compilation
13118
7ff40da08c2a towards 1.6b1
Fredrik Lundh <fredrik@pythonware.com>
parents: 12955
diff changeset
127
7ff40da08c2a towards 1.6b1
Fredrik Lundh <fredrik@pythonware.com>
parents: 12955
diff changeset
128 # sre exception
13136
b6ed8fa2675c - fixed split
Fredrik Lundh <fredrik@pythonware.com>
parents: 13118
diff changeset
129 error = sre_compile.error
13118
7ff40da08c2a towards 1.6b1
Fredrik Lundh <fredrik@pythonware.com>
parents: 12955
diff changeset
130
12175
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast.
Guido van Rossum <guido@python.org>
parents:
diff changeset
131 # --------------------------------------------------------------------
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast.
Guido van Rossum <guido@python.org>
parents:
diff changeset
132 # public interface
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast.
Guido van Rossum <guido@python.org>
parents:
diff changeset
133
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast.
Guido van Rossum <guido@python.org>
parents:
diff changeset
134 def match(pattern, string, flags=0):
15855
152d00718006 SRE fixes for 2.1 alpha:
Fredrik Lundh <fredrik@pythonware.com>
parents: 14726
diff changeset
135 """Try to apply the pattern at the start of the string, returning
152d00718006 SRE fixes for 2.1 alpha:
Fredrik Lundh <fredrik@pythonware.com>
parents: 14726
diff changeset
136 a match object, or None if no match was found."""
12819
a24d9ad6786d Fredrik Lundh: here's the 96.6% version of SRE
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 12548
diff changeset
137 return _compile(pattern, flags).match(string)
12175
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast.
Guido van Rossum <guido@python.org>
parents:
diff changeset
138
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast.
Guido van Rossum <guido@python.org>
parents:
diff changeset
139 def search(pattern, string, flags=0):
15855
152d00718006 SRE fixes for 2.1 alpha:
Fredrik Lundh <fredrik@pythonware.com>
parents: 14726
diff changeset
140 """Scan through string looking for a match to the pattern, returning
152d00718006 SRE fixes for 2.1 alpha:
Fredrik Lundh <fredrik@pythonware.com>
parents: 14726
diff changeset
141 a match object, or None if no match was found."""
12819
a24d9ad6786d Fredrik Lundh: here's the 96.6% version of SRE
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 12548
diff changeset
142 return _compile(pattern, flags).search(string)
a24d9ad6786d Fredrik Lundh: here's the 96.6% version of SRE
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 12548
diff changeset
143
51936
8f5d8f27c6fc Adds an optional flags argument to re.split, re.sub and re.subn to be
Gregory P. Smith <greg@mad-scientist.com>
parents: 50424
diff changeset
144 def sub(pattern, repl, string, count=0, flags=0):
15855
152d00718006 SRE fixes for 2.1 alpha:
Fredrik Lundh <fredrik@pythonware.com>
parents: 14726
diff changeset
145 """Return the string obtained by replacing the leftmost
152d00718006 SRE fixes for 2.1 alpha:
Fredrik Lundh <fredrik@pythonware.com>
parents: 14726
diff changeset
146 non-overlapping occurrences of the pattern in string by the
33011
ff17624b2642 [Bug #990792] Mention that repl can be a callable
Andrew M. Kuchling <amk@amk.ca>
parents: 31934
diff changeset
147 replacement repl. repl can be either a string or a callable;
50424
df3923c36ec5 Merged revisions 67952-67953,67955,67957-67958,67960-67961,67963,67965,67967,67970-67971,67973,67982,67988,67990,67995,68014,68016,68030,68057,68061,68112,68115-68118,68120-68121,68123-68128 via svnmerge from
Georg Brandl <georg@python.org>
parents: 48171
diff changeset
148 if a string, backslash escapes in it are processed. If it is
df3923c36ec5 Merged revisions 67952-67953,67955,67957-67958,67960-67961,67963,67965,67967,67970-67971,67973,67982,67988,67990,67995,68014,68016,68030,68057,68061,68112,68115-68118,68120-68121,68123-68128 via svnmerge from
Georg Brandl <georg@python.org>
parents: 48171
diff changeset
149 a callable, it's passed the match object and must return
33011
ff17624b2642 [Bug #990792] Mention that repl can be a callable
Andrew M. Kuchling <amk@amk.ca>
parents: 31934
diff changeset
150 a replacement string to be used."""
51936
8f5d8f27c6fc Adds an optional flags argument to re.split, re.sub and re.subn to be
Gregory P. Smith <greg@mad-scientist.com>
parents: 50424
diff changeset
151 return _compile(pattern, flags).sub(repl, string, count)
12819
a24d9ad6786d Fredrik Lundh: here's the 96.6% version of SRE
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 12548
diff changeset
152
51936
8f5d8f27c6fc Adds an optional flags argument to re.split, re.sub and re.subn to be
Gregory P. Smith <greg@mad-scientist.com>
parents: 50424
diff changeset
153 def subn(pattern, repl, string, count=0, flags=0):
15855
152d00718006 SRE fixes for 2.1 alpha:
Fredrik Lundh <fredrik@pythonware.com>
parents: 14726
diff changeset
154 """Return a 2-tuple containing (new_string, number).
152d00718006 SRE fixes for 2.1 alpha:
Fredrik Lundh <fredrik@pythonware.com>
parents: 14726
diff changeset
155 new_string is the string obtained by replacing the leftmost
152d00718006 SRE fixes for 2.1 alpha:
Fredrik Lundh <fredrik@pythonware.com>
parents: 14726
diff changeset
156 non-overlapping occurrences of the pattern in the source
152d00718006 SRE fixes for 2.1 alpha:
Fredrik Lundh <fredrik@pythonware.com>
parents: 14726
diff changeset
157 string by the replacement repl. number is the number of
33011
ff17624b2642 [Bug #990792] Mention that repl can be a callable
Andrew M. Kuchling <amk@amk.ca>
parents: 31934
diff changeset
158 substitutions that were made. repl can be either a string or a
50424
df3923c36ec5 Merged revisions 67952-67953,67955,67957-67958,67960-67961,67963,67965,67967,67970-67971,67973,67982,67988,67990,67995,68014,68016,68030,68057,68061,68112,68115-68118,68120-68121,68123-68128 via svnmerge from
Georg Brandl <georg@python.org>
parents: 48171
diff changeset
159 callable; if a string, backslash escapes in it are processed.
df3923c36ec5 Merged revisions 67952-67953,67955,67957-67958,67960-67961,67963,67965,67967,67970-67971,67973,67982,67988,67990,67995,68014,68016,68030,68057,68061,68112,68115-68118,68120-68121,68123-68128 via svnmerge from
Georg Brandl <georg@python.org>
parents: 48171
diff changeset
160 If it is a callable, it's passed the match object and must
33011
ff17624b2642 [Bug #990792] Mention that repl can be a callable
Andrew M. Kuchling <amk@amk.ca>
parents: 31934
diff changeset
161 return a replacement string to be used."""
51936
8f5d8f27c6fc Adds an optional flags argument to re.split, re.sub and re.subn to be
Gregory P. Smith <greg@mad-scientist.com>
parents: 50424
diff changeset
162 return _compile(pattern, flags).subn(repl, string, count)
12819
a24d9ad6786d Fredrik Lundh: here's the 96.6% version of SRE
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 12548
diff changeset
163
51936
8f5d8f27c6fc Adds an optional flags argument to re.split, re.sub and re.subn to be
Gregory P. Smith <greg@mad-scientist.com>
parents: 50424
diff changeset
164 def split(pattern, string, maxsplit=0, flags=0):
15855
152d00718006 SRE fixes for 2.1 alpha:
Fredrik Lundh <fredrik@pythonware.com>
parents: 14726
diff changeset
165 """Split the source string by the occurrences of the pattern,
152d00718006 SRE fixes for 2.1 alpha:
Fredrik Lundh <fredrik@pythonware.com>
parents: 14726
diff changeset
166 returning a list containing the resulting substrings."""
51936
8f5d8f27c6fc Adds an optional flags argument to re.split, re.sub and re.subn to be
Gregory P. Smith <greg@mad-scientist.com>
parents: 50424
diff changeset
167 return _compile(pattern, flags).split(string, maxsplit)
12819
a24d9ad6786d Fredrik Lundh: here's the 96.6% version of SRE
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 12548
diff changeset
168
33552
bf9f917ee4f9 Granted Noam Raphael's request for minor improvements to the re module and
Raymond Hettinger <python@rcn.com>
parents: 33257
diff changeset
169 def findall(pattern, string, flags=0):
15855
152d00718006 SRE fixes for 2.1 alpha:
Fredrik Lundh <fredrik@pythonware.com>
parents: 14726
diff changeset
170 """Return a list of all non-overlapping matches in the string.
152d00718006 SRE fixes for 2.1 alpha:
Fredrik Lundh <fredrik@pythonware.com>
parents: 14726
diff changeset
171
152d00718006 SRE fixes for 2.1 alpha:
Fredrik Lundh <fredrik@pythonware.com>
parents: 14726
diff changeset
172 If one or more groups are present in the pattern, return a
152d00718006 SRE fixes for 2.1 alpha:
Fredrik Lundh <fredrik@pythonware.com>
parents: 14726
diff changeset
173 list of groups; this will be a list of tuples if the pattern
152d00718006 SRE fixes for 2.1 alpha:
Fredrik Lundh <fredrik@pythonware.com>
parents: 14726
diff changeset
174 has more than one group.
152d00718006 SRE fixes for 2.1 alpha:
Fredrik Lundh <fredrik@pythonware.com>
parents: 14726
diff changeset
175
152d00718006 SRE fixes for 2.1 alpha:
Fredrik Lundh <fredrik@pythonware.com>
parents: 14726
diff changeset
176 Empty matches are included in the result."""
33552
bf9f917ee4f9 Granted Noam Raphael's request for minor improvements to the re module and
Raymond Hettinger <python@rcn.com>
parents: 33257
diff changeset
177 return _compile(pattern, flags).findall(string)
12175
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast.
Guido van Rossum <guido@python.org>
parents:
diff changeset
178
20660
c05d4895234c (experimental) "finditer" method/function. this works pretty much
Fredrik Lundh <fredrik@pythonware.com>
parents: 20592
diff changeset
179 if sys.hexversion >= 0x02020000:
25759
5632841a32a4 Add finditer to __all__ (when defining it at all).
Guido van Rossum <guido@python.org>
parents: 20731
diff changeset
180 __all__.append("finditer")
33552
bf9f917ee4f9 Granted Noam Raphael's request for minor improvements to the re module and
Raymond Hettinger <python@rcn.com>
parents: 33257
diff changeset
181 def finditer(pattern, string, flags=0):
20731
8359c1189f73 added finditer sanity check
Fredrik Lundh <fredrik@pythonware.com>
parents: 20660
diff changeset
182 """Return an iterator over all non-overlapping matches in the
8359c1189f73 added finditer sanity check
Fredrik Lundh <fredrik@pythonware.com>
parents: 20660
diff changeset
183 string. For each match, the iterator returns a match object.
20660
c05d4895234c (experimental) "finditer" method/function. this works pretty much
Fredrik Lundh <fredrik@pythonware.com>
parents: 20592
diff changeset
184
c05d4895234c (experimental) "finditer" method/function. this works pretty much
Fredrik Lundh <fredrik@pythonware.com>
parents: 20592
diff changeset
185 Empty matches are included in the result."""
33552
bf9f917ee4f9 Granted Noam Raphael's request for minor improvements to the re module and
Raymond Hettinger <python@rcn.com>
parents: 33257
diff changeset
186 return _compile(pattern, flags).finditer(string)
20660
c05d4895234c (experimental) "finditer" method/function. this works pretty much
Fredrik Lundh <fredrik@pythonware.com>
parents: 20592
diff changeset
187
12819
a24d9ad6786d Fredrik Lundh: here's the 96.6% version of SRE
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 12548
diff changeset
188 def compile(pattern, flags=0):
15855
152d00718006 SRE fixes for 2.1 alpha:
Fredrik Lundh <fredrik@pythonware.com>
parents: 14726
diff changeset
189 "Compile a regular expression pattern, returning a pattern object."
12819
a24d9ad6786d Fredrik Lundh: here's the 96.6% version of SRE
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 12548
diff changeset
190 return _compile(pattern, flags)
a24d9ad6786d Fredrik Lundh: here's the 96.6% version of SRE
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 12548
diff changeset
191
13677
edb1a8e6b859 -- SRE 0.9.6 sync. this includes:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13311
diff changeset
192 def purge():
15855
152d00718006 SRE fixes for 2.1 alpha:
Fredrik Lundh <fredrik@pythonware.com>
parents: 14726
diff changeset
193 "Clear the regular expression cache"
13677
edb1a8e6b859 -- SRE 0.9.6 sync. this includes:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13311
diff changeset
194 _cache.clear()
17144
bbf5ea2a22e5 sre 2.1b2 update:
Fredrik Lundh <fredrik@pythonware.com>
parents: 16663
diff changeset
195 _cache_repl.clear()
13677
edb1a8e6b859 -- SRE 0.9.6 sync. this includes:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13311
diff changeset
196
13118
7ff40da08c2a towards 1.6b1
Fredrik Lundh <fredrik@pythonware.com>
parents: 12955
diff changeset
197 def template(pattern, flags=0):
15855
152d00718006 SRE fixes for 2.1 alpha:
Fredrik Lundh <fredrik@pythonware.com>
parents: 14726
diff changeset
198 "Compile a template pattern, returning a pattern object"
13118
7ff40da08c2a towards 1.6b1
Fredrik Lundh <fredrik@pythonware.com>
parents: 12955
diff changeset
199 return _compile(pattern, flags|T)
7ff40da08c2a towards 1.6b1
Fredrik Lundh <fredrik@pythonware.com>
parents: 12955
diff changeset
200
75113
475d5a749756 a frozenset is better here
Benjamin Peterson <benjamin@python.org>
parents: 75112
diff changeset
201 _alphanum = frozenset(
75122
32c8af00eef2 remove extra zero
Benjamin Peterson <benjamin@python.org>
parents: 75113
diff changeset
202 "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789")
35540
e41fda9f0fd1 Speed-up escape()
Raymond Hettinger <python@rcn.com>
parents: 33552
diff changeset
203
12819
a24d9ad6786d Fredrik Lundh: here's the 96.6% version of SRE
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 12548
diff changeset
204 def escape(pattern):
15855
152d00718006 SRE fixes for 2.1 alpha:
Fredrik Lundh <fredrik@pythonware.com>
parents: 14726
diff changeset
205 "Escape all non-alphanumeric characters in pattern."
12819
a24d9ad6786d Fredrik Lundh: here's the 96.6% version of SRE
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 12548
diff changeset
206 s = list(pattern)
35540
e41fda9f0fd1 Speed-up escape()
Raymond Hettinger <python@rcn.com>
parents: 33552
diff changeset
207 alphanum = _alphanum
68927
d52b1faa7b11 #2650: Refactor re.escape and its tests.
Ezio Melotti <ezio.melotti@gmail.com>
parents: 68256
diff changeset
208 for i, c in enumerate(pattern):
35540
e41fda9f0fd1 Speed-up escape()
Raymond Hettinger <python@rcn.com>
parents: 33552
diff changeset
209 if c not in alphanum:
12819
a24d9ad6786d Fredrik Lundh: here's the 96.6% version of SRE
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 12548
diff changeset
210 if c == "\000":
a24d9ad6786d Fredrik Lundh: here's the 96.6% version of SRE
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 12548
diff changeset
211 s[i] = "\\000"
a24d9ad6786d Fredrik Lundh: here's the 96.6% version of SRE
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 12548
diff changeset
212 else:
a24d9ad6786d Fredrik Lundh: here's the 96.6% version of SRE
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 12548
diff changeset
213 s[i] = "\\" + c
33257
11a4fbc88a80 PEP 292 classes Template and SafeTemplate are added to the string module.
Barry Warsaw <barry@python.org>
parents: 33011
diff changeset
214 return pattern[:0].join(s)
12175
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast.
Guido van Rossum <guido@python.org>
parents:
diff changeset
215
0155a3b13bf0 Added Fredrik Lundh's sre module and its supporting cast.
Guido van Rossum <guido@python.org>
parents:
diff changeset
216 # --------------------------------------------------------------------
12819
a24d9ad6786d Fredrik Lundh: here's the 96.6% version of SRE
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 12548
diff changeset
217 # internals
a24d9ad6786d Fredrik Lundh: here's the 96.6% version of SRE
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 12548
diff changeset
218
a24d9ad6786d Fredrik Lundh: here's the 96.6% version of SRE
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 12548
diff changeset
219 _cache = {}
17144
bbf5ea2a22e5 sre 2.1b2 update:
Fredrik Lundh <fredrik@pythonware.com>
parents: 16663
diff changeset
220 _cache_repl = {}
bbf5ea2a22e5 sre 2.1b2 update:
Fredrik Lundh <fredrik@pythonware.com>
parents: 16663
diff changeset
221
20534
c2005bb879b4 SRE bug #441409:
Fredrik Lundh <fredrik@pythonware.com>
parents: 19971
diff changeset
222 _pattern_type = type(sre_compile.compile("", 0))
c2005bb879b4 SRE bug #441409:
Fredrik Lundh <fredrik@pythonware.com>
parents: 19971
diff changeset
223
12819
a24d9ad6786d Fredrik Lundh: here's the 96.6% version of SRE
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 12548
diff changeset
224 _MAXCACHE = 100
a24d9ad6786d Fredrik Lundh: here's the 96.6% version of SRE
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 12548
diff changeset
225
13886
62494a96dcca -- reset marks if repeat_one tail doesn't match
Fredrik Lundh <fredrik@pythonware.com>
parents: 13795
diff changeset
226 def _compile(*key):
12819
a24d9ad6786d Fredrik Lundh: here's the 96.6% version of SRE
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 12548
diff changeset
227 # internal: compile pattern
31934
fdabb6808d52 SF #926075: Fixed the bug that returns a wrong pattern object for
Hye-Shik Chang <hyeshik@gmail.com>
parents: 29749
diff changeset
228 cachekey = (type(key[0]),) + key
fdabb6808d52 SF #926075: Fixed the bug that returns a wrong pattern object for
Hye-Shik Chang <hyeshik@gmail.com>
parents: 29749
diff changeset
229 p = _cache.get(cachekey)
13886
62494a96dcca -- reset marks if repeat_one tail doesn't match
Fredrik Lundh <fredrik@pythonware.com>
parents: 13795
diff changeset
230 if p is not None:
62494a96dcca -- reset marks if repeat_one tail doesn't match
Fredrik Lundh <fredrik@pythonware.com>
parents: 13795
diff changeset
231 return p
62494a96dcca -- reset marks if repeat_one tail doesn't match
Fredrik Lundh <fredrik@pythonware.com>
parents: 13795
diff changeset
232 pattern, flags = key
29748
5af89037ceba Fix and test for bug #764548:
Just van Rossum <just@letterror.com>
parents: 25759
diff changeset
233 if isinstance(pattern, _pattern_type):
44354
b1347c68d456 Fix issue 1661: Flags argument silently ignored in re functions with compiled regexes.
Raymond Hettinger <python@rcn.com>
parents: 37017
diff changeset
234 if flags:
b1347c68d456 Fix issue 1661: Flags argument silently ignored in re functions with compiled regexes.
Raymond Hettinger <python@rcn.com>
parents: 37017
diff changeset
235 raise ValueError('Cannot process flags argument with a compiled pattern')
20534
c2005bb879b4 SRE bug #441409:
Fredrik Lundh <fredrik@pythonware.com>
parents: 19971
diff changeset
236 return pattern
29749
ac9a1f0a33dc Addendum to #764548: restore 2.1 compatibility.
Just van Rossum <just@letterror.com>
parents: 29748
diff changeset
237 if not sre_compile.isstring(pattern):
20534
c2005bb879b4 SRE bug #441409:
Fredrik Lundh <fredrik@pythonware.com>
parents: 19971
diff changeset
238 raise TypeError, "first argument must be string or compiled pattern"
13795
8295f8506eae final 0.9.8 updates:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13789
diff changeset
239 try:
8295f8506eae final 0.9.8 updates:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13789
diff changeset
240 p = sre_compile.compile(pattern, flags)
8295f8506eae final 0.9.8 updates:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13789
diff changeset
241 except error, v:
8295f8506eae final 0.9.8 updates:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13789
diff changeset
242 raise error, v # invalid expression
12819
a24d9ad6786d Fredrik Lundh: here's the 96.6% version of SRE
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 12548
diff changeset
243 if len(_cache) >= _MAXCACHE:
a24d9ad6786d Fredrik Lundh: here's the 96.6% version of SRE
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 12548
diff changeset
244 _cache.clear()
31934
fdabb6808d52 SF #926075: Fixed the bug that returns a wrong pattern object for
Hye-Shik Chang <hyeshik@gmail.com>
parents: 29749
diff changeset
245 _cache[cachekey] = p
12819
a24d9ad6786d Fredrik Lundh: here's the 96.6% version of SRE
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 12548
diff changeset
246 return p
a24d9ad6786d Fredrik Lundh: here's the 96.6% version of SRE
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 12548
diff changeset
247
17144
bbf5ea2a22e5 sre 2.1b2 update:
Fredrik Lundh <fredrik@pythonware.com>
parents: 16663
diff changeset
248 def _compile_repl(*key):
bbf5ea2a22e5 sre 2.1b2 update:
Fredrik Lundh <fredrik@pythonware.com>
parents: 16663
diff changeset
249 # internal: compile replacement pattern
bbf5ea2a22e5 sre 2.1b2 update:
Fredrik Lundh <fredrik@pythonware.com>
parents: 16663
diff changeset
250 p = _cache_repl.get(key)
bbf5ea2a22e5 sre 2.1b2 update:
Fredrik Lundh <fredrik@pythonware.com>
parents: 16663
diff changeset
251 if p is not None:
bbf5ea2a22e5 sre 2.1b2 update:
Fredrik Lundh <fredrik@pythonware.com>
parents: 16663
diff changeset
252 return p
bbf5ea2a22e5 sre 2.1b2 update:
Fredrik Lundh <fredrik@pythonware.com>
parents: 16663
diff changeset
253 repl, pattern = key
bbf5ea2a22e5 sre 2.1b2 update:
Fredrik Lundh <fredrik@pythonware.com>
parents: 16663
diff changeset
254 try:
bbf5ea2a22e5 sre 2.1b2 update:
Fredrik Lundh <fredrik@pythonware.com>
parents: 16663
diff changeset
255 p = sre_parse.parse_template(repl, pattern)
bbf5ea2a22e5 sre 2.1b2 update:
Fredrik Lundh <fredrik@pythonware.com>
parents: 16663
diff changeset
256 except error, v:
bbf5ea2a22e5 sre 2.1b2 update:
Fredrik Lundh <fredrik@pythonware.com>
parents: 16663
diff changeset
257 raise error, v # invalid expression
bbf5ea2a22e5 sre 2.1b2 update:
Fredrik Lundh <fredrik@pythonware.com>
parents: 16663
diff changeset
258 if len(_cache_repl) >= _MAXCACHE:
bbf5ea2a22e5 sre 2.1b2 update:
Fredrik Lundh <fredrik@pythonware.com>
parents: 16663
diff changeset
259 _cache_repl.clear()
bbf5ea2a22e5 sre 2.1b2 update:
Fredrik Lundh <fredrik@pythonware.com>
parents: 16663
diff changeset
260 _cache_repl[key] = p
bbf5ea2a22e5 sre 2.1b2 update:
Fredrik Lundh <fredrik@pythonware.com>
parents: 16663
diff changeset
261 return p
bbf5ea2a22e5 sre 2.1b2 update:
Fredrik Lundh <fredrik@pythonware.com>
parents: 16663
diff changeset
262
14726
176660ff0c95 - fixed yet another gcc -pedantic warning
Fredrik Lundh <fredrik@pythonware.com>
parents: 13886
diff changeset
263 def _expand(pattern, match, template):
176660ff0c95 - fixed yet another gcc -pedantic warning
Fredrik Lundh <fredrik@pythonware.com>
parents: 13886
diff changeset
264 # internal: match.expand implementation hook
176660ff0c95 - fixed yet another gcc -pedantic warning
Fredrik Lundh <fredrik@pythonware.com>
parents: 13886
diff changeset
265 template = sre_parse.parse_template(template, pattern)
176660ff0c95 - fixed yet another gcc -pedantic warning
Fredrik Lundh <fredrik@pythonware.com>
parents: 13886
diff changeset
266 return sre_parse.expand_template(template, match)
176660ff0c95 - fixed yet another gcc -pedantic warning
Fredrik Lundh <fredrik@pythonware.com>
parents: 13886
diff changeset
267
20589
d3298435d85b rewrote the pattern.sub and pattern.subn methods in C
Fredrik Lundh <fredrik@pythonware.com>
parents: 20534
diff changeset
268 def _subx(pattern, template):
d3298435d85b rewrote the pattern.sub and pattern.subn methods in C
Fredrik Lundh <fredrik@pythonware.com>
parents: 20534
diff changeset
269 # internal: pattern.sub/subn implementation helper
20592
82bcfac89642 fixed character set description in docstring (SRE uses Python
Fredrik Lundh <fredrik@pythonware.com>
parents: 20590
diff changeset
270 template = _compile_repl(template, pattern)
82bcfac89642 fixed character set description in docstring (SRE uses Python
Fredrik Lundh <fredrik@pythonware.com>
parents: 20590
diff changeset
271 if not template[0] and len(template[1]) == 1:
20589
d3298435d85b rewrote the pattern.sub and pattern.subn methods in C
Fredrik Lundh <fredrik@pythonware.com>
parents: 20534
diff changeset
272 # literal replacement
20592
82bcfac89642 fixed character set description in docstring (SRE uses Python
Fredrik Lundh <fredrik@pythonware.com>
parents: 20590
diff changeset
273 return template[1][0]
82bcfac89642 fixed character set description in docstring (SRE uses Python
Fredrik Lundh <fredrik@pythonware.com>
parents: 20590
diff changeset
274 def filter(match, template=template):
82bcfac89642 fixed character set description in docstring (SRE uses Python
Fredrik Lundh <fredrik@pythonware.com>
parents: 20590
diff changeset
275 return sre_parse.expand_template(template, match)
82bcfac89642 fixed character set description in docstring (SRE uses Python
Fredrik Lundh <fredrik@pythonware.com>
parents: 20590
diff changeset
276 return filter
13201
a093fad92ccf the mad patcher strikes again:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13186
diff changeset
277
a093fad92ccf the mad patcher strikes again:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13186
diff changeset
278 # register myself for pickling
a093fad92ccf the mad patcher strikes again:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13186
diff changeset
279
a093fad92ccf the mad patcher strikes again:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13186
diff changeset
280 import copy_reg
a093fad92ccf the mad patcher strikes again:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13186
diff changeset
281
a093fad92ccf the mad patcher strikes again:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13186
diff changeset
282 def _pickle(p):
a093fad92ccf the mad patcher strikes again:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13186
diff changeset
283 return _compile, (p.pattern, p.flags)
a093fad92ccf the mad patcher strikes again:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13186
diff changeset
284
20534
c2005bb879b4 SRE bug #441409:
Fredrik Lundh <fredrik@pythonware.com>
parents: 19971
diff changeset
285 copy_reg.pickle(_pattern_type, _pickle, _compile)
13308
fc2ad6fa08ec - actually enabled charset anchors in the engine (still not
Fredrik Lundh <fredrik@pythonware.com>
parents: 13299
diff changeset
286
fc2ad6fa08ec - actually enabled charset anchors in the engine (still not
Fredrik Lundh <fredrik@pythonware.com>
parents: 13299
diff changeset
287 # --------------------------------------------------------------------
fc2ad6fa08ec - actually enabled charset anchors in the engine (still not
Fredrik Lundh <fredrik@pythonware.com>
parents: 13299
diff changeset
288 # experimental stuff (see python-dev discussions for details)
fc2ad6fa08ec - actually enabled charset anchors in the engine (still not
Fredrik Lundh <fredrik@pythonware.com>
parents: 13299
diff changeset
289
fc2ad6fa08ec - actually enabled charset anchors in the engine (still not
Fredrik Lundh <fredrik@pythonware.com>
parents: 13299
diff changeset
290 class Scanner:
20590
050ed8a21cde sre.Scanner fixes (from Greg Chapman). also added a Scanner sanity
Fredrik Lundh <fredrik@pythonware.com>
parents: 20589
diff changeset
291 def __init__(self, lexicon, flags=0):
13789
4edd13eb759d SRE 0.9.8: passes the entire test suite
Fredrik Lundh <fredrik@pythonware.com>
parents: 13677
diff changeset
292 from sre_constants import BRANCH, SUBPATTERN
13308
fc2ad6fa08ec - actually enabled charset anchors in the engine (still not
Fredrik Lundh <fredrik@pythonware.com>
parents: 13299
diff changeset
293 self.lexicon = lexicon
13677
edb1a8e6b859 -- SRE 0.9.6 sync. this includes:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13311
diff changeset
294 # combine phrases into a compound pattern
13308
fc2ad6fa08ec - actually enabled charset anchors in the engine (still not
Fredrik Lundh <fredrik@pythonware.com>
parents: 13299
diff changeset
295 p = []
13677
edb1a8e6b859 -- SRE 0.9.6 sync. this includes:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13311
diff changeset
296 s = sre_parse.Pattern()
20590
050ed8a21cde sre.Scanner fixes (from Greg Chapman). also added a Scanner sanity
Fredrik Lundh <fredrik@pythonware.com>
parents: 20589
diff changeset
297 s.flags = flags
13308
fc2ad6fa08ec - actually enabled charset anchors in the engine (still not
Fredrik Lundh <fredrik@pythonware.com>
parents: 13299
diff changeset
298 for phrase, action in lexicon:
13677
edb1a8e6b859 -- SRE 0.9.6 sync. this includes:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13311
diff changeset
299 p.append(sre_parse.SubPattern(s, [
20590
050ed8a21cde sre.Scanner fixes (from Greg Chapman). also added a Scanner sanity
Fredrik Lundh <fredrik@pythonware.com>
parents: 20589
diff changeset
300 (SUBPATTERN, (len(p)+1, sre_parse.parse(phrase, flags))),
13677
edb1a8e6b859 -- SRE 0.9.6 sync. this includes:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13311
diff changeset
301 ]))
45170
4d58c0fa6e34 In the experimental 'Scanner' feature, the group count was set wrong.
Guido van Rossum <guido@python.org>
parents: 37017
diff changeset
302 s.groups = len(p)+1
13677
edb1a8e6b859 -- SRE 0.9.6 sync. this includes:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13311
diff changeset
303 p = sre_parse.SubPattern(s, [(BRANCH, (None, p))])
edb1a8e6b859 -- SRE 0.9.6 sync. this includes:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13311
diff changeset
304 self.scanner = sre_compile.compile(p)
13308
fc2ad6fa08ec - actually enabled charset anchors in the engine (still not
Fredrik Lundh <fredrik@pythonware.com>
parents: 13299
diff changeset
305 def scan(self, string):
fc2ad6fa08ec - actually enabled charset anchors in the engine (still not
Fredrik Lundh <fredrik@pythonware.com>
parents: 13299
diff changeset
306 result = []
fc2ad6fa08ec - actually enabled charset anchors in the engine (still not
Fredrik Lundh <fredrik@pythonware.com>
parents: 13299
diff changeset
307 append = result.append
20590
050ed8a21cde sre.Scanner fixes (from Greg Chapman). also added a Scanner sanity
Fredrik Lundh <fredrik@pythonware.com>
parents: 20589
diff changeset
308 match = self.scanner.scanner(string).match
13308
fc2ad6fa08ec - actually enabled charset anchors in the engine (still not
Fredrik Lundh <fredrik@pythonware.com>
parents: 13299
diff changeset
309 i = 0
fc2ad6fa08ec - actually enabled charset anchors in the engine (still not
Fredrik Lundh <fredrik@pythonware.com>
parents: 13299
diff changeset
310 while 1:
20590
050ed8a21cde sre.Scanner fixes (from Greg Chapman). also added a Scanner sanity
Fredrik Lundh <fredrik@pythonware.com>
parents: 20589
diff changeset
311 m = match()
13308
fc2ad6fa08ec - actually enabled charset anchors in the engine (still not
Fredrik Lundh <fredrik@pythonware.com>
parents: 13299
diff changeset
312 if not m:
fc2ad6fa08ec - actually enabled charset anchors in the engine (still not
Fredrik Lundh <fredrik@pythonware.com>
parents: 13299
diff changeset
313 break
fc2ad6fa08ec - actually enabled charset anchors in the engine (still not
Fredrik Lundh <fredrik@pythonware.com>
parents: 13299
diff changeset
314 j = m.end()
fc2ad6fa08ec - actually enabled charset anchors in the engine (still not
Fredrik Lundh <fredrik@pythonware.com>
parents: 13299
diff changeset
315 if i == j:
fc2ad6fa08ec - actually enabled charset anchors in the engine (still not
Fredrik Lundh <fredrik@pythonware.com>
parents: 13299
diff changeset
316 break
20590
050ed8a21cde sre.Scanner fixes (from Greg Chapman). also added a Scanner sanity
Fredrik Lundh <fredrik@pythonware.com>
parents: 20589
diff changeset
317 action = self.lexicon[m.lastindex-1][1]
48171
6bc3cfe098ce Remove the use of callable() in re to silence warnings under -3.
Brett Cannon <bcannon@gmail.com>
parents: 47189
diff changeset
318 if hasattr(action, '__call__'):
15855
152d00718006 SRE fixes for 2.1 alpha:
Fredrik Lundh <fredrik@pythonware.com>
parents: 14726
diff changeset
319 self.match = m
13308
fc2ad6fa08ec - actually enabled charset anchors in the engine (still not
Fredrik Lundh <fredrik@pythonware.com>
parents: 13299
diff changeset
320 action = action(self, m.group())
fc2ad6fa08ec - actually enabled charset anchors in the engine (still not
Fredrik Lundh <fredrik@pythonware.com>
parents: 13299
diff changeset
321 if action is not None:
fc2ad6fa08ec - actually enabled charset anchors in the engine (still not
Fredrik Lundh <fredrik@pythonware.com>
parents: 13299
diff changeset
322 append(action)
fc2ad6fa08ec - actually enabled charset anchors in the engine (still not
Fredrik Lundh <fredrik@pythonware.com>
parents: 13299
diff changeset
323 i = j
fc2ad6fa08ec - actually enabled charset anchors in the engine (still not
Fredrik Lundh <fredrik@pythonware.com>
parents: 13299
diff changeset
324 return result, string[i:]