Mercurial > cpython-lru_cache-weakref

annotate Doc/library/stringprep.rst @ 87690:66c1c9f32567 default tip

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Add use_weakref option to functools.lru_cache The use_weakref option makes lru_cache create weak references to the function arguments when creating the cache key, which prevents them from staying in the memory forever even after all other references to them are gone. This is a proof of concept for http://bugs.python.org/issue19859
author Radomir Dopieralski <devel@sheep.art.pl>
date Tue, 03 Dec 2013 16:02:48 +0100
parents fa91270b6087
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
42604
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
1 :mod:`stringprep` --- Internet String Preparation
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
2 =================================================
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
3
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
4 .. module:: stringprep
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
5 :synopsis: String preparation, as per RFC 3453
42630
20aec9f34598 Manually patched a few things that didn't get merged in, but should.
Guido van Rossum <guido@python.org>
parents: 42604
diff changeset
6 :deprecated:
42604
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
7 .. moduleauthor:: Martin v. Löwis <martin@v.loewis.de>
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
8 .. sectionauthor:: Martin v. Löwis <martin@v.loewis.de>
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
9
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
10
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
11 When identifying things (such as host names) in the internet, it is often
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
12 necessary to compare such identifications for "equality". Exactly how this
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
13 comparison is executed may depend on the application domain, e.g. whether it
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
14 should be case-insensitive or not. It may be also necessary to restrict the
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
15 possible identifications, to allow only identifications consisting of
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
16 "printable" characters.
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
17
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
18 :rfc:`3454` defines a procedure for "preparing" Unicode strings in internet
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
19 protocols. Before passing strings onto the wire, they are processed with the
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
20 preparation procedure, after which they have a certain normalized form. The RFC
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
21 defines a set of tables, which can be combined into profiles. Each profile must
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
22 define which tables it uses, and what other optional parts of the ``stringprep``
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
23 procedure are part of the profile. One example of a ``stringprep`` profile is
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
24 ``nameprep``, which is used for internationalized domain names.
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
25
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
26 The module :mod:`stringprep` only exposes the tables from RFC 3454. As these
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
27 tables would be very large to represent them as dictionaries or lists, the
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
28 module uses the Unicode character database internally. The module source code
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
29 itself was generated using the ``mkstringprep.py`` utility.
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
30
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
31 As a result, these tables are exposed as functions, not as data structures.
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
32 There are two kinds of tables in the RFC: sets and mappings. For a set,
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
33 :mod:`stringprep` provides the "characteristic function", i.e. a function that
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
34 returns true if the parameter is part of the set. For mappings, it provides the
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
35 mapping function: given the key, it returns the associated value. Below is a
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
36 list of all functions available in the module.
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
37
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
38
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
39 .. function:: in_table_a1(code)
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
40
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
41 Determine whether *code* is in tableA.1 (Unassigned code points in Unicode 3.2).
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
42
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
43
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
44 .. function:: in_table_b1(code)
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
45
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
46 Determine whether *code* is in tableB.1 (Commonly mapped to nothing).
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
47
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
48
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
49 .. function:: map_table_b2(code)
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
50
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
51 Return the mapped value for *code* according to tableB.2 (Mapping for
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
52 case-folding used with NFKC).
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
53
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
54
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
55 .. function:: map_table_b3(code)
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
56
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
57 Return the mapped value for *code* according to tableB.3 (Mapping for
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
58 case-folding used with no normalization).
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
59
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
60
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
61 .. function:: in_table_c11(code)
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
62
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
63 Determine whether *code* is in tableC.1.1 (ASCII space characters).
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
64
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
65
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
66 .. function:: in_table_c12(code)
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
67
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
68 Determine whether *code* is in tableC.1.2 (Non-ASCII space characters).
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
69
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
70
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
71 .. function:: in_table_c11_c12(code)
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
72
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
73 Determine whether *code* is in tableC.1 (Space characters, union of C.1.1 and
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
74 C.1.2).
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
75
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
76
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
77 .. function:: in_table_c21(code)
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
78
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
79 Determine whether *code* is in tableC.2.1 (ASCII control characters).
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
80
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
81
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
82 .. function:: in_table_c22(code)
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
83
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
84 Determine whether *code* is in tableC.2.2 (Non-ASCII control characters).
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
85
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
86
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
87 .. function:: in_table_c21_c22(code)
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
88
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
89 Determine whether *code* is in tableC.2 (Control characters, union of C.2.1 and
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
90 C.2.2).
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
91
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
92
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
93 .. function:: in_table_c3(code)
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
94
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
95 Determine whether *code* is in tableC.3 (Private use).
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
96
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
97
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
98 .. function:: in_table_c4(code)
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
99
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
100 Determine whether *code* is in tableC.4 (Non-character code points).
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
101
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
102
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
103 .. function:: in_table_c5(code)
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
104
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
105 Determine whether *code* is in tableC.5 (Surrogate codes).
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
106
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
107
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
108 .. function:: in_table_c6(code)
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
109
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
110 Determine whether *code* is in tableC.6 (Inappropriate for plain text).
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
111
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
112
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
113 .. function:: in_table_c7(code)
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
114
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
115 Determine whether *code* is in tableC.7 (Inappropriate for canonical
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
116 representation).
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
117
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
118
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
119 .. function:: in_table_c8(code)
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
120
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
121 Determine whether *code* is in tableC.8 (Change display properties or are
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
122 deprecated).
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
123
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
124
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
125 .. function:: in_table_c9(code)
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
126
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
127 Determine whether *code* is in tableC.9 (Tagging characters).
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
128
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
129
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
130 .. function:: in_table_d1(code)
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
131
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
132 Determine whether *code* is in tableD.1 (Characters with bidirectional property
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
133 "R" or "AL").
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
134
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
135
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
136 .. function:: in_table_d2(code)
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
137
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
138 Determine whether *code* is in tableD.2 (Characters with bidirectional property
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
139 "L").
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
140