Mercurial > cpython-lru_cache-weakref

annotate Doc/library/stringprep.rst @ 78069:a8aa77458e12 3.2

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Add c_size_t and c_ssize_t to the ctypes summary table.
author Antoine Pitrou <solipsis@pitrou.net>
date Thu, 12 Jul 2012 20:31:50 +0200
parents fa91270b6087
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
42604
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
1 :mod:`stringprep` --- Internet String Preparation
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
2 =================================================
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
3
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
4 .. module:: stringprep
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
5 :synopsis: String preparation, as per RFC 3453
42630
20aec9f34598 Manually patched a few things that didn't get merged in, but should.
Guido van Rossum <guido@python.org>
parents: 42604
diff changeset
6 :deprecated:
42604
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
7 .. moduleauthor:: Martin v. Löwis <martin@v.loewis.de>
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
8 .. sectionauthor:: Martin v. Löwis <martin@v.loewis.de>
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
9
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
10
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
11 When identifying things (such as host names) in the internet, it is often
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
12 necessary to compare such identifications for "equality". Exactly how this
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
13 comparison is executed may depend on the application domain, e.g. whether it
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
14 should be case-insensitive or not. It may be also necessary to restrict the
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
15 possible identifications, to allow only identifications consisting of
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
16 "printable" characters.
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
17
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
18 :rfc:`3454` defines a procedure for "preparing" Unicode strings in internet
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
19 protocols. Before passing strings onto the wire, they are processed with the
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
20 preparation procedure, after which they have a certain normalized form. The RFC
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
21 defines a set of tables, which can be combined into profiles. Each profile must
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
22 define which tables it uses, and what other optional parts of the ``stringprep``
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
23 procedure are part of the profile. One example of a ``stringprep`` profile is
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
24 ``nameprep``, which is used for internationalized domain names.
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
25
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
26 The module :mod:`stringprep` only exposes the tables from RFC 3454. As these
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
27 tables would be very large to represent them as dictionaries or lists, the
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
28 module uses the Unicode character database internally. The module source code
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
29 itself was generated using the ``mkstringprep.py`` utility.
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
30
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
31 As a result, these tables are exposed as functions, not as data structures.
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
32 There are two kinds of tables in the RFC: sets and mappings. For a set,
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
33 :mod:`stringprep` provides the "characteristic function", i.e. a function that
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
34 returns true if the parameter is part of the set. For mappings, it provides the
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
35 mapping function: given the key, it returns the associated value. Below is a
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
36 list of all functions available in the module.
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
37
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
38
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
39 .. function:: in_table_a1(code)
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
40
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
41 Determine whether *code* is in tableA.1 (Unassigned code points in Unicode 3.2).
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
42
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
43
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
44 .. function:: in_table_b1(code)
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
45
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
46 Determine whether *code* is in tableB.1 (Commonly mapped to nothing).
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
47
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
48
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
49 .. function:: map_table_b2(code)
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
50
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
51 Return the mapped value for *code* according to tableB.2 (Mapping for
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
52 case-folding used with NFKC).
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
53
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
54
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
55 .. function:: map_table_b3(code)
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
56
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
57 Return the mapped value for *code* according to tableB.3 (Mapping for
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
58 case-folding used with no normalization).
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
59
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
60
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
61 .. function:: in_table_c11(code)
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
62
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
63 Determine whether *code* is in tableC.1.1 (ASCII space characters).
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
64
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
65
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
66 .. function:: in_table_c12(code)
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
67
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
68 Determine whether *code* is in tableC.1.2 (Non-ASCII space characters).
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
69
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
70
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
71 .. function:: in_table_c11_c12(code)
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
72
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
73 Determine whether *code* is in tableC.1 (Space characters, union of C.1.1 and
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
74 C.1.2).
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
75
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
76
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
77 .. function:: in_table_c21(code)
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
78
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
79 Determine whether *code* is in tableC.2.1 (ASCII control characters).
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
80
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
81
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
82 .. function:: in_table_c22(code)
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
83
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
84 Determine whether *code* is in tableC.2.2 (Non-ASCII control characters).
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
85
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
86
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
87 .. function:: in_table_c21_c22(code)
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
88
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
89 Determine whether *code* is in tableC.2 (Control characters, union of C.2.1 and
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
90 C.2.2).
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
91
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
92
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
93 .. function:: in_table_c3(code)
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
94
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
95 Determine whether *code* is in tableC.3 (Private use).
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
96
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
97
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
98 .. function:: in_table_c4(code)
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
99
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
100 Determine whether *code* is in tableC.4 (Non-character code points).
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
101
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
102
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
103 .. function:: in_table_c5(code)
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
104
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
105 Determine whether *code* is in tableC.5 (Surrogate codes).
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
106
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
107
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
108 .. function:: in_table_c6(code)
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
109
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
110 Determine whether *code* is in tableC.6 (Inappropriate for plain text).
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
111
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
112
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
113 .. function:: in_table_c7(code)
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
114
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
115 Determine whether *code* is in tableC.7 (Inappropriate for canonical
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
116 representation).
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
117
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
118
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
119 .. function:: in_table_c8(code)
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
120
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
121 Determine whether *code* is in tableC.8 (Change display properties or are
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
122 deprecated).
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
123
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
124
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
125 .. function:: in_table_c9(code)
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
126
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
127 Determine whether *code* is in tableC.9 (Tagging characters).
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
128
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
129
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
130 .. function:: in_table_d1(code)
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
131
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
132 Determine whether *code* is in tableD.1 (Characters with bidirectional property
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
133 "R" or "AL").
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
134
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
135
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
136 .. function:: in_table_d2(code)
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
137
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
138 Determine whether *code* is in tableD.2 (Characters with bidirectional property
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
139 "L").
9e1529bf0442 Move the 3k reST doc tree in place.
Georg Brandl <georg@python.org>
parents:
diff changeset
140