Mercurial > cpython-lru_cache-weakref

annotate Lib/re.py @ 78069:a8aa77458e12 3.2

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Add c_size_t and c_ssize_t to the ctypes summary table.
author Antoine Pitrou <solipsis@pitrou.net>
date Thu, 12 Jul 2012 20:31:50 +0200
parents 23d5b457dc71
children 14fe7a98b89c 9165774a8055
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
37877
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
1 #
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
2 # Secret Labs' Regular Expression Engine
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
3 #
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
4 # re-compatible interface for the sre matching engine
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
5 #
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
6 # Copyright (c) 1998-2001 by Secret Labs AB. All rights reserved.
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
7 #
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
8 # This version of the SRE library can be redistributed under CNRI's
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
9 # Python 1.6 license. For any other use, please contact Secret Labs
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
10 # AB (info@pythonware.com).
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
11 #
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
12 # Portions of this engine have been developed in cooperation with
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
13 # CNRI. Hewlett-Packard provided funding for 1.6 integration and
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
14 # other compatibility work.
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
15 #
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
16
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
17 r"""Support for regular expressions (RE).
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
18
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
19 This module provides regular expression matching operations similar to
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
20 those found in Perl. It supports both 8-bit and Unicode strings; both
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
21 the pattern and the strings being processed can contain null bytes and
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
22 characters outside the US ASCII range.
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
23
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
24 Regular expressions can contain both special and ordinary characters.
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
25 Most ordinary characters, like "A", "a", or "0", are the simplest
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
26 regular expressions; they simply match themselves. You can
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
27 concatenate ordinary characters, so last matches the string 'last'.
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
28
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
29 The special characters are:
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
30 "." Matches any character except a newline.
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
31 "^" Matches the start of the string.
44859
feba5d5cac22 Merged revisions 60151-60159,60161-60168,60170,60172-60173,60175 via svnmerge from
Georg Brandl <georg@python.org>
parents: 44373
diff changeset
32 "$" Matches the end of the string or just before the newline at
feba5d5cac22 Merged revisions 60151-60159,60161-60168,60170,60172-60173,60175 via svnmerge from
Georg Brandl <georg@python.org>
parents: 44373
diff changeset
33 the end of the string.
37877
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
34 "*" Matches 0 or more (greedy) repetitions of the preceding RE.
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
35 Greedy means that it will match as many repetitions as possible.
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
36 "+" Matches 1 or more (greedy) repetitions of the preceding RE.
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
37 "?" Matches 0 or 1 (greedy) of the preceding RE.
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
38 *?,+?,?? Non-greedy versions of the previous three special characters.
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
39 {m,n} Matches from m to n repetitions of the preceding RE.
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
40 {m,n}? Non-greedy version of the above.
44999
7c02182e2979 Merged revisions 60284-60349 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents: 44859
diff changeset
41 "\\" Either escapes special characters or signals a special sequence.
37877
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
42 [] Indicates a set of characters.
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
43 A "^" as the first character indicates a complementing set.
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
44 "|" A|B, creates an RE that will match either A or B.
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
45 (...) Matches the RE inside the parentheses.
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
46 The contents can be retrieved or matched later in the string.
48769
83b2d07b475b #2834: Change re module semantics, so that str and bytes mixing is forbidden,
Antoine Pitrou <solipsis@pitrou.net>
parents: 46867
diff changeset
47 (?aiLmsux) Set the A, I, L, M, S, U, or X flag for the RE (see below).
37877
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
48 (?:...) Non-grouping version of regular parentheses.
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
49 (?P<name>...) The substring matched by the group is accessible by name.
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
50 (?P=name) Matches the text matched earlier by the group named name.
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
51 (?#...) A comment; ignored.
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
52 (?=...) Matches if ... matches next, but doesn't consume the string.
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
53 (?!...) Matches if ... doesn't match next.
44999
7c02182e2979 Merged revisions 60284-60349 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents: 44859
diff changeset
54 (?<=...) Matches if preceded by ... (must be fixed length).
7c02182e2979 Merged revisions 60284-60349 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents: 44859
diff changeset
55 (?<!...) Matches if not preceded by ... (must be fixed length).
7c02182e2979 Merged revisions 60284-60349 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents: 44859
diff changeset
56 (?(id/name)yes|no) Matches yes pattern if the group with id/name matched,
7c02182e2979 Merged revisions 60284-60349 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents: 44859
diff changeset
57 the (optional) no pattern otherwise.
37877
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
58
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
59 The special sequences consist of "\\" and a character from the list
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
60 below. If the ordinary character is not on the list, then the
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
61 resulting RE will match the second character.
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
62 \number Matches the contents of the group of the same number.
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
63 \A Matches only at the start of the string.
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
64 \Z Matches only at the end of the string.
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
65 \b Matches the empty string, but only at the start or end of a word.
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
66 \B Matches the empty string, but not at the start or end of a word.
48769
83b2d07b475b #2834: Change re module semantics, so that str and bytes mixing is forbidden,
Antoine Pitrou <solipsis@pitrou.net>
parents: 46867
diff changeset
67 \d Matches any decimal digit; equivalent to the set [0-9] in
83b2d07b475b #2834: Change re module semantics, so that str and bytes mixing is forbidden,
Antoine Pitrou <solipsis@pitrou.net>
parents: 46867
diff changeset
68 bytes patterns or string patterns with the ASCII flag.
83b2d07b475b #2834: Change re module semantics, so that str and bytes mixing is forbidden,
Antoine Pitrou <solipsis@pitrou.net>
parents: 46867
diff changeset
69 In string patterns without the ASCII flag, it will match the whole
83b2d07b475b #2834: Change re module semantics, so that str and bytes mixing is forbidden,
Antoine Pitrou <solipsis@pitrou.net>
parents: 46867
diff changeset
70 range of Unicode digits.
83b2d07b475b #2834: Change re module semantics, so that str and bytes mixing is forbidden,
Antoine Pitrou <solipsis@pitrou.net>
parents: 46867
diff changeset
71 \D Matches any non-digit character; equivalent to [^\d].
76615
da9179248118 #14236: mention Unicode whitespace in \s documentation.
Ezio Melotti <ezio.melotti@gmail.com>
parents: 75513
diff changeset
72 \s Matches any whitespace character; equivalent to [ \t\n\r\f\v] in
da9179248118 #14236: mention Unicode whitespace in \s documentation.
Ezio Melotti <ezio.melotti@gmail.com>
parents: 75513
diff changeset
73 bytes patterns or string patterns with the ASCII flag.
da9179248118 #14236: mention Unicode whitespace in \s documentation.
Ezio Melotti <ezio.melotti@gmail.com>
parents: 75513
diff changeset
74 In string patterns without the ASCII flag, it will match the whole
da9179248118 #14236: mention Unicode whitespace in \s documentation.
Ezio Melotti <ezio.melotti@gmail.com>
parents: 75513
diff changeset
75 range of Unicode whitespace characters.
76627
23d5b457dc71 #14236: fix docs for \S.
Ezio Melotti <ezio.melotti@gmail.com>
parents: 76615
diff changeset
76 \S Matches any non-whitespace character; equivalent to [^\s].
48769
83b2d07b475b #2834: Change re module semantics, so that str and bytes mixing is forbidden,
Antoine Pitrou <solipsis@pitrou.net>
parents: 46867
diff changeset
77 \w Matches any alphanumeric character; equivalent to [a-zA-Z0-9_]
83b2d07b475b #2834: Change re module semantics, so that str and bytes mixing is forbidden,
Antoine Pitrou <solipsis@pitrou.net>
parents: 46867
diff changeset
78 in bytes patterns or string patterns with the ASCII flag.
83b2d07b475b #2834: Change re module semantics, so that str and bytes mixing is forbidden,
Antoine Pitrou <solipsis@pitrou.net>
parents: 46867
diff changeset
79 In string patterns without the ASCII flag, it will match the
83b2d07b475b #2834: Change re module semantics, so that str and bytes mixing is forbidden,
Antoine Pitrou <solipsis@pitrou.net>
parents: 46867
diff changeset
80 range of Unicode alphanumeric characters (letters plus digits
83b2d07b475b #2834: Change re module semantics, so that str and bytes mixing is forbidden,
Antoine Pitrou <solipsis@pitrou.net>
parents: 46867
diff changeset
81 plus underscore).
37877
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
82 With LOCALE, it will match the set [0-9_] plus characters defined
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
83 as letters for the current locale.
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
84 \W Matches the complement of \w.
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
85 \\ Matches a literal backslash.
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
86
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
87 This module exports the following functions:
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
88 match Match a regular expression pattern to the beginning of a string.
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
89 search Search a string for the presence of a pattern.
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
90 sub Substitute occurrences of a pattern found in a string.
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
91 subn Same as sub, but also return the number of substitutions made.
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
92 split Split a string by the occurrences of a pattern.
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
93 findall Find all occurrences of a pattern in a string.
44999
7c02182e2979 Merged revisions 60284-60349 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents: 44859
diff changeset
94 finditer Return an iterator yielding a match object for each match.
37877
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
95 compile Compile a pattern into a RegexObject.
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
96 purge Clear the regular expression cache.
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
97 escape Backslash all non-alphanumerics in a string.
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
98
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
99 Some of the functions in this module takes flags as optional parameters:
48769
83b2d07b475b #2834: Change re module semantics, so that str and bytes mixing is forbidden,
Antoine Pitrou <solipsis@pitrou.net>
parents: 46867
diff changeset
100 A ASCII For string patterns, make \w, \W, \b, \B, \d, \D
83b2d07b475b #2834: Change re module semantics, so that str and bytes mixing is forbidden,
Antoine Pitrou <solipsis@pitrou.net>
parents: 46867
diff changeset
101 match the corresponding ASCII character categories
83b2d07b475b #2834: Change re module semantics, so that str and bytes mixing is forbidden,
Antoine Pitrou <solipsis@pitrou.net>
parents: 46867
diff changeset
102 (rather than the whole Unicode categories, which is the
83b2d07b475b #2834: Change re module semantics, so that str and bytes mixing is forbidden,
Antoine Pitrou <solipsis@pitrou.net>
parents: 46867
diff changeset
103 default).
83b2d07b475b #2834: Change re module semantics, so that str and bytes mixing is forbidden,
Antoine Pitrou <solipsis@pitrou.net>
parents: 46867
diff changeset
104 For bytes patterns, this flag is the only available
83b2d07b475b #2834: Change re module semantics, so that str and bytes mixing is forbidden,
Antoine Pitrou <solipsis@pitrou.net>
parents: 46867
diff changeset
105 behaviour and needn't be specified.
37877
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
106 I IGNORECASE Perform case-insensitive matching.
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
107 L LOCALE Make \w, \W, \b, \B, dependent on the current locale.
44859
feba5d5cac22 Merged revisions 60151-60159,60161-60168,60170,60172-60173,60175 via svnmerge from
Georg Brandl <georg@python.org>
parents: 44373
diff changeset
108 M MULTILINE "^" matches the beginning of lines (after a newline)
feba5d5cac22 Merged revisions 60151-60159,60161-60168,60170,60172-60173,60175 via svnmerge from
Georg Brandl <georg@python.org>
parents: 44373
diff changeset
109 as well as the string.
feba5d5cac22 Merged revisions 60151-60159,60161-60168,60170,60172-60173,60175 via svnmerge from
Georg Brandl <georg@python.org>
parents: 44373
diff changeset
110 "$" matches the end of lines (before a newline) as well
feba5d5cac22 Merged revisions 60151-60159,60161-60168,60170,60172-60173,60175 via svnmerge from
Georg Brandl <georg@python.org>
parents: 44373
diff changeset
111 as the end of the string.
37877
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
112 S DOTALL "." matches any character at all, including the newline.
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
113 X VERBOSE Ignore whitespace and comments for nicer looking RE's.
48769
83b2d07b475b #2834: Change re module semantics, so that str and bytes mixing is forbidden,
Antoine Pitrou <solipsis@pitrou.net>
parents: 46867
diff changeset
114 U UNICODE For compatibility only. Ignored for string patterns (it
83b2d07b475b #2834: Change re module semantics, so that str and bytes mixing is forbidden,
Antoine Pitrou <solipsis@pitrou.net>
parents: 46867
diff changeset
115 is the default), and forbidden for bytes patterns.
37877
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
116
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
117 This module also defines an exception 'error'.
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
118
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
119 """
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
120
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
121 import sys
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
122 import sre_compile
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
123 import sre_parse
63953
f1d922d820ca Reapply r83877.
Antoine Pitrou <solipsis@pitrou.net>
parents: 63871
diff changeset
124 import functools
37877
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
125
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
126 # public symbols
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
127 __all__ = [ "match", "search", "sub", "subn", "split", "findall",
48769
83b2d07b475b #2834: Change re module semantics, so that str and bytes mixing is forbidden,
Antoine Pitrou <solipsis@pitrou.net>
parents: 46867
diff changeset
128 "compile", "purge", "template", "escape", "A", "I", "L", "M", "S", "X",
83b2d07b475b #2834: Change re module semantics, so that str and bytes mixing is forbidden,
Antoine Pitrou <solipsis@pitrou.net>
parents: 46867
diff changeset
129 "U", "ASCII", "IGNORECASE", "LOCALE", "MULTILINE", "DOTALL", "VERBOSE",
37877
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
130 "UNICODE", "error" ]
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
131
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
132 __version__ = "2.2.1"
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
133
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
134 # flags
48769
83b2d07b475b #2834: Change re module semantics, so that str and bytes mixing is forbidden,
Antoine Pitrou <solipsis@pitrou.net>
parents: 46867
diff changeset
135 A = ASCII = sre_compile.SRE_FLAG_ASCII # assume ascii "locale"
37877
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
136 I = IGNORECASE = sre_compile.SRE_FLAG_IGNORECASE # ignore case
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
137 L = LOCALE = sre_compile.SRE_FLAG_LOCALE # assume current 8-bit locale
48769
83b2d07b475b #2834: Change re module semantics, so that str and bytes mixing is forbidden,
Antoine Pitrou <solipsis@pitrou.net>
parents: 46867
diff changeset
138 U = UNICODE = sre_compile.SRE_FLAG_UNICODE # assume unicode "locale"
37877
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
139 M = MULTILINE = sre_compile.SRE_FLAG_MULTILINE # make anchors look for newline
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
140 S = DOTALL = sre_compile.SRE_FLAG_DOTALL # make dot match newline
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
141 X = VERBOSE = sre_compile.SRE_FLAG_VERBOSE # ignore whitespace and comments
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
142
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
143 # sre extensions (experimental, don't rely on these)
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
144 T = TEMPLATE = sre_compile.SRE_FLAG_TEMPLATE # disable backtracking
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
145 DEBUG = sre_compile.SRE_FLAG_DEBUG # dump pattern after compilation
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
146
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
147 # sre exception
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
148 error = sre_compile.error
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
149
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
150 # --------------------------------------------------------------------
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
151 # public interface
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
152
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
153 def match(pattern, string, flags=0):
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
154 """Try to apply the pattern at the start of the string, returning
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
155 a match object, or None if no match was found."""
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
156 return _compile(pattern, flags).match(string)
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
157
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
158 def search(pattern, string, flags=0):
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
159 """Scan through string looking for a match to the pattern, returning
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
160 a match object, or None if no match was found."""
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
161 return _compile(pattern, flags).search(string)
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
162
51937
dcd3bc2e2377 Merged revisions 70090 via svnmerge from
Gregory P. Smith <greg@mad-scientist.com>
parents: 50419
diff changeset
163 def sub(pattern, repl, string, count=0, flags=0):
37877
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
164 """Return the string obtained by replacing the leftmost
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
165 non-overlapping occurrences of the pattern in string by the
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
166 replacement repl. repl can be either a string or a callable;
50419
8463ff861a26 Merged revisions 68129 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 49128
diff changeset
167 if a string, backslash escapes in it are processed. If it is
8463ff861a26 Merged revisions 68129 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 49128
diff changeset
168 a callable, it's passed the match object and must return
37877
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
169 a replacement string to be used."""
51937
dcd3bc2e2377 Merged revisions 70090 via svnmerge from
Gregory P. Smith <greg@mad-scientist.com>
parents: 50419
diff changeset
170 return _compile(pattern, flags).sub(repl, string, count)
37877
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
171
51937
dcd3bc2e2377 Merged revisions 70090 via svnmerge from
Gregory P. Smith <greg@mad-scientist.com>
parents: 50419
diff changeset
172 def subn(pattern, repl, string, count=0, flags=0):
37877
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
173 """Return a 2-tuple containing (new_string, number).
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
174 new_string is the string obtained by replacing the leftmost
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
175 non-overlapping occurrences of the pattern in the source
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
176 string by the replacement repl. number is the number of
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
177 substitutions that were made. repl can be either a string or a
50419
8463ff861a26 Merged revisions 68129 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 49128
diff changeset
178 callable; if a string, backslash escapes in it are processed.
8463ff861a26 Merged revisions 68129 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 49128
diff changeset
179 If it is a callable, it's passed the match object and must
37877
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
180 return a replacement string to be used."""
51937
dcd3bc2e2377 Merged revisions 70090 via svnmerge from
Gregory P. Smith <greg@mad-scientist.com>
parents: 50419
diff changeset
181 return _compile(pattern, flags).subn(repl, string, count)
37877
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
182
51937
dcd3bc2e2377 Merged revisions 70090 via svnmerge from
Gregory P. Smith <greg@mad-scientist.com>
parents: 50419
diff changeset
183 def split(pattern, string, maxsplit=0, flags=0):
37877
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
184 """Split the source string by the occurrences of the pattern,
75513
b96251d9f36a Closes #14244: add info about capturing groups and maxsplit to the docstring of re.split().
Georg Brandl <georg@python.org>
parents: 73165
diff changeset
185 returning a list containing the resulting substrings. If
b96251d9f36a Closes #14244: add info about capturing groups and maxsplit to the docstring of re.split().
Georg Brandl <georg@python.org>
parents: 73165
diff changeset
186 capturing parentheses are used in pattern, then the text of all
b96251d9f36a Closes #14244: add info about capturing groups and maxsplit to the docstring of re.split().
Georg Brandl <georg@python.org>
parents: 73165
diff changeset
187 groups in the pattern are also returned as part of the resulting
b96251d9f36a Closes #14244: add info about capturing groups and maxsplit to the docstring of re.split().
Georg Brandl <georg@python.org>
parents: 73165
diff changeset
188 list. If maxsplit is nonzero, at most maxsplit splits occur,
b96251d9f36a Closes #14244: add info about capturing groups and maxsplit to the docstring of re.split().
Georg Brandl <georg@python.org>
parents: 73165
diff changeset
189 and the remainder of the string is returned as the final element
b96251d9f36a Closes #14244: add info about capturing groups and maxsplit to the docstring of re.split().
Georg Brandl <georg@python.org>
parents: 73165
diff changeset
190 of the list."""
51937
dcd3bc2e2377 Merged revisions 70090 via svnmerge from
Gregory P. Smith <greg@mad-scientist.com>
parents: 50419
diff changeset
191 return _compile(pattern, flags).split(string, maxsplit)
37877
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
192
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
193 def findall(pattern, string, flags=0):
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
194 """Return a list of all non-overlapping matches in the string.
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
195
75513
b96251d9f36a Closes #14244: add info about capturing groups and maxsplit to the docstring of re.split().
Georg Brandl <georg@python.org>
parents: 73165
diff changeset
196 If one or more capturing groups are present in the pattern, return
b96251d9f36a Closes #14244: add info about capturing groups and maxsplit to the docstring of re.split().
Georg Brandl <georg@python.org>
parents: 73165
diff changeset
197 a list of groups; this will be a list of tuples if the pattern
37877
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
198 has more than one group.
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
199
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
200 Empty matches are included in the result."""
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
201 return _compile(pattern, flags).findall(string)
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
202
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
203 if sys.hexversion >= 0x02020000:
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
204 __all__.append("finditer")
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
205 def finditer(pattern, string, flags=0):
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
206 """Return an iterator over all non-overlapping matches in the
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
207 string. For each match, the iterator returns a match object.
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
208
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
209 Empty matches are included in the result."""
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
210 return _compile(pattern, flags).finditer(string)
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
211
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
212 def compile(pattern, flags=0):
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
213 "Compile a regular expression pattern, returning a pattern object."
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
214 return _compile(pattern, flags)
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
215
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
216 def purge():
63953
f1d922d820ca Reapply r83877.
Antoine Pitrou <solipsis@pitrou.net>
parents: 63871
diff changeset
217 "Clear the regular expression caches"
64448
61c44793238d Adopt more descriptive attribute names as suggested on python-dev.
Raymond Hettinger <python@rcn.com>
parents: 64014
diff changeset
218 _compile_typed.cache_clear()
61c44793238d Adopt more descriptive attribute names as suggested on python-dev.
Raymond Hettinger <python@rcn.com>
parents: 64014
diff changeset
219 _compile_repl.cache_clear()
37877
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
220
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
221 def template(pattern, flags=0):
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
222 "Compile a template pattern, returning a pattern object"
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
223 return _compile(pattern, flags|T)
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
224
49128
513266a75ec3 Issue #3756: make re.escape() handle bytes as well as str.
Guido van Rossum <guido@python.org>
parents: 48769
diff changeset
225 _alphanum_str = frozenset(
513266a75ec3 Issue #3756: make re.escape() handle bytes as well as str.
Guido van Rossum <guido@python.org>
parents: 48769
diff changeset
226 "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890")
513266a75ec3 Issue #3756: make re.escape() handle bytes as well as str.
Guido van Rossum <guido@python.org>
parents: 48769
diff changeset
227 _alphanum_bytes = frozenset(
513266a75ec3 Issue #3756: make re.escape() handle bytes as well as str.
Guido van Rossum <guido@python.org>
parents: 48769
diff changeset
228 b"abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890")
37877
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
229
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
230 def escape(pattern):
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
231 "Escape all non-alphanumeric characters in pattern."
49128
513266a75ec3 Issue #3756: make re.escape() handle bytes as well as str.
Guido van Rossum <guido@python.org>
parents: 48769
diff changeset
232 if isinstance(pattern, str):
513266a75ec3 Issue #3756: make re.escape() handle bytes as well as str.
Guido van Rossum <guido@python.org>
parents: 48769
diff changeset
233 alphanum = _alphanum_str
513266a75ec3 Issue #3756: make re.escape() handle bytes as well as str.
Guido van Rossum <guido@python.org>
parents: 48769
diff changeset
234 s = list(pattern)
68924
ed02db9921ac #2650: Refactor re.escape to use enumerate().
Ezio Melotti <ezio.melotti@gmail.com>
parents: 51937
diff changeset
235 for i, c in enumerate(pattern):
49128
513266a75ec3 Issue #3756: make re.escape() handle bytes as well as str.
Guido van Rossum <guido@python.org>
parents: 48769
diff changeset
236 if c not in alphanum:
513266a75ec3 Issue #3756: make re.escape() handle bytes as well as str.
Guido van Rossum <guido@python.org>
parents: 48769
diff changeset
237 if c == "\000":
513266a75ec3 Issue #3756: make re.escape() handle bytes as well as str.
Guido van Rossum <guido@python.org>
parents: 48769
diff changeset
238 s[i] = "\\000"
513266a75ec3 Issue #3756: make re.escape() handle bytes as well as str.
Guido van Rossum <guido@python.org>
parents: 48769
diff changeset
239 else:
513266a75ec3 Issue #3756: make re.escape() handle bytes as well as str.
Guido van Rossum <guido@python.org>
parents: 48769
diff changeset
240 s[i] = "\\" + c
513266a75ec3 Issue #3756: make re.escape() handle bytes as well as str.
Guido van Rossum <guido@python.org>
parents: 48769
diff changeset
241 return "".join(s)
513266a75ec3 Issue #3756: make re.escape() handle bytes as well as str.
Guido van Rossum <guido@python.org>
parents: 48769
diff changeset
242 else:
513266a75ec3 Issue #3756: make re.escape() handle bytes as well as str.
Guido van Rossum <guido@python.org>
parents: 48769
diff changeset
243 alphanum = _alphanum_bytes
513266a75ec3 Issue #3756: make re.escape() handle bytes as well as str.
Guido van Rossum <guido@python.org>
parents: 48769
diff changeset
244 s = []
513266a75ec3 Issue #3756: make re.escape() handle bytes as well as str.
Guido van Rossum <guido@python.org>
parents: 48769
diff changeset
245 esc = ord(b"\\")
513266a75ec3 Issue #3756: make re.escape() handle bytes as well as str.
Guido van Rossum <guido@python.org>
parents: 48769
diff changeset
246 for c in pattern:
513266a75ec3 Issue #3756: make re.escape() handle bytes as well as str.
Guido van Rossum <guido@python.org>
parents: 48769
diff changeset
247 if c in alphanum:
513266a75ec3 Issue #3756: make re.escape() handle bytes as well as str.
Guido van Rossum <guido@python.org>
parents: 48769
diff changeset
248 s.append(c)
37877
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
249 else:
49128
513266a75ec3 Issue #3756: make re.escape() handle bytes as well as str.
Guido van Rossum <guido@python.org>
parents: 48769
diff changeset
250 if c == 0:
513266a75ec3 Issue #3756: make re.escape() handle bytes as well as str.
Guido van Rossum <guido@python.org>
parents: 48769
diff changeset
251 s.extend(b"\\000")
513266a75ec3 Issue #3756: make re.escape() handle bytes as well as str.
Guido van Rossum <guido@python.org>
parents: 48769
diff changeset
252 else:
513266a75ec3 Issue #3756: make re.escape() handle bytes as well as str.
Guido van Rossum <guido@python.org>
parents: 48769
diff changeset
253 s.append(esc)
513266a75ec3 Issue #3756: make re.escape() handle bytes as well as str.
Guido van Rossum <guido@python.org>
parents: 48769
diff changeset
254 s.append(c)
513266a75ec3 Issue #3756: make re.escape() handle bytes as well as str.
Guido van Rossum <guido@python.org>
parents: 48769
diff changeset
255 return bytes(s)
37877
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
256
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
257 # --------------------------------------------------------------------
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
258 # internals
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
259
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
260 _pattern_type = type(sre_compile.compile("", 0))
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
261
64014
3a1b5b1c069b Clean-up the cache patch for the re module.
Raymond Hettinger <python@rcn.com>
parents: 63953
diff changeset
262 def _compile(pattern, flags):
3a1b5b1c069b Clean-up the cache patch for the re module.
Raymond Hettinger <python@rcn.com>
parents: 63953
diff changeset
263 return _compile_typed(type(pattern), pattern, flags)
37877
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
264
64014
3a1b5b1c069b Clean-up the cache patch for the re module.
Raymond Hettinger <python@rcn.com>
parents: 63953
diff changeset
265 @functools.lru_cache(maxsize=500)
3a1b5b1c069b Clean-up the cache patch for the re module.
Raymond Hettinger <python@rcn.com>
parents: 63953
diff changeset
266 def _compile_typed(text_bytes_type, pattern, flags):
37877
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
267 # internal: compile pattern
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
268 if isinstance(pattern, _pattern_type):
44373
2750d531fdc0 Merged revisions 59565-59594 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents: 43020
diff changeset
269 if flags:
49128
513266a75ec3 Issue #3756: make re.escape() handle bytes as well as str.
Guido van Rossum <guido@python.org>
parents: 48769
diff changeset
270 raise ValueError(
513266a75ec3 Issue #3756: make re.escape() handle bytes as well as str.
Guido van Rossum <guido@python.org>
parents: 48769
diff changeset
271 "Cannot process flags argument with a compiled pattern")
37877
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
272 return pattern
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
273 if not sre_compile.isstring(pattern):
43020
907a45df1295 Raise statement normalization in Lib/.
Collin Winter <collinw@gmail.com>
parents: 41824
diff changeset
274 raise TypeError("first argument must be string or compiled pattern")
63953
f1d922d820ca Reapply r83877.
Antoine Pitrou <solipsis@pitrou.net>
parents: 63871
diff changeset
275 return sre_compile.compile(pattern, flags)
37877
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
276
64014
3a1b5b1c069b Clean-up the cache patch for the re module.
Raymond Hettinger <python@rcn.com>
parents: 63953
diff changeset
277 @functools.lru_cache(maxsize=500)
3a1b5b1c069b Clean-up the cache patch for the re module.
Raymond Hettinger <python@rcn.com>
parents: 63953
diff changeset
278 def _compile_repl(repl, pattern):
37877
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
279 # internal: compile replacement pattern
63953
f1d922d820ca Reapply r83877.
Antoine Pitrou <solipsis@pitrou.net>
parents: 63871
diff changeset
280 return sre_parse.parse_template(repl, pattern)
37877
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
281
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
282 def _expand(pattern, match, template):
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
283 # internal: match.expand implementation hook
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
284 template = sre_parse.parse_template(template, pattern)
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
285 return sre_parse.expand_template(template, match)
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
286
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
287 def _subx(pattern, template):
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
288 # internal: pattern.sub/subn implementation helper
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
289 template = _compile_repl(template, pattern)
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
290 if not template[0] and len(template[1]) == 1:
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
291 # literal replacement
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
292 return template[1][0]
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
293 def filter(match, template=template):
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
294 return sre_parse.expand_template(template, match)
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
295 return filter
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
296
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
297 # register myself for pickling
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
298
46867
dd5513b472dc Rename copy_reg module to copyreg.
Alexandre Vassalotti <alexandre@peadrop.com>
parents: 45192
diff changeset
299 import copyreg
37877
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
300
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
301 def _pickle(p):
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
302 return _compile, (p.pattern, p.flags)
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
303
46867
dd5513b472dc Rename copy_reg module to copyreg.
Alexandre Vassalotti <alexandre@peadrop.com>
parents: 45192
diff changeset
304 copyreg.pickle(_pattern_type, _pickle, _compile)
37877
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
305
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
306 # --------------------------------------------------------------------
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
307 # experimental stuff (see python-dev discussions for details)
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
308
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
309 class Scanner:
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
310 def __init__(self, lexicon, flags=0):
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
311 from sre_constants import BRANCH, SUBPATTERN
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
312 self.lexicon = lexicon
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
313 # combine phrases into a compound pattern
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
314 p = []
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
315 s = sre_parse.Pattern()
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
316 s.flags = flags
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
317 for phrase, action in lexicon:
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
318 p.append(sre_parse.SubPattern(s, [
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
319 (SUBPATTERN, (len(p)+1, sre_parse.parse(phrase, flags))),
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
320 ]))
45192
142a11b3057e Merged revisions 60481,60485,60489-60492,60494-60496,60498-60499,60501-60503,60505-60506,60508-60509,60523-60524,60532,60543,60545,60547-60548,60552,60554,60556-60559,60561-60562,60568-60598,60600-60616 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents: 44999
diff changeset
321 s.groups = len(p)+1
37877
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
322 p = sre_parse.SubPattern(s, [(BRANCH, (None, p))])
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
323 self.scanner = sre_compile.compile(p)
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
324 def scan(self, string):
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
325 result = []
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
326 append = result.append
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
327 match = self.scanner.scanner(string).match
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
328 i = 0
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
329 while 1:
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
330 m = match()
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
331 if not m:
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
332 break
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
333 j = m.end()
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
334 if i == j:
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
335 break
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
336 action = self.lexicon[m.lastindex-1][1]
73165
8e57b5d8f58f Closes #13258: Use callable() built-in in the standard library.
Florent Xicluna <florent.xicluna@gmail.com>
parents: 68925
diff changeset
337 if callable(action):
37877
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
338 self.match = m
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
339 action = action(self, m.group())
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
340 if action is not None:
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
341 append(action)
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
342 i = j
6a45f26ea619 Merge trunk up to 43069, putting re.py back and hopefully making the branch
Thomas Wouters <thomas@python.org>
parents:
diff changeset
343 return result, string[i:]