Mercurial > cpython-lru_cache-weakref

annotate Lib/urlparse.py @ 58765:2ba7677bfa95 legacy-trunk

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
testCopysign was defined twice in test_math; combine the definitions
author Mark Dickinson <dickinsm@gmail.com>
date Sat, 06 Feb 2010 23:11:25 +0000
parents 2ff89f0f1d19
children 7da7f9bfdaac
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
11707
6f4146846337 The third and final doc-string sweep by Ka-Ping Yee.
Guido van Rossum <guido@python.org>
parents: 10830
diff changeset
1 """Parse (absolute and relative) URLs.
6f4146846337 The third and final doc-string sweep by Ka-Ping Yee.
Guido van Rossum <guido@python.org>
parents: 10830
diff changeset
2
6f4146846337 The third and final doc-string sweep by Ka-Ping Yee.
Guido van Rossum <guido@python.org>
parents: 10830
diff changeset
3 See RFC 1808: "Relative Uniform Resource Locators", by R. Fielding,
6f4146846337 The third and final doc-string sweep by Ka-Ping Yee.
Guido van Rossum <guido@python.org>
parents: 10830
diff changeset
4 UC Irvine, June 1995.
6f4146846337 The third and final doc-string sweep by Ka-Ping Yee.
Guido van Rossum <guido@python.org>
parents: 10830
diff changeset
5 """
1875
8c9cc054c5ed New tty/pty modules by Steen; new urlparser.
Guido van Rossum <guido@python.org>
parents:
diff changeset
6
25814
b8cbdb735cf2 Added missing entries to __all__.
Fred Drake <fdrake@acm.org>
parents: 25787
diff changeset
7 __all__ = ["urlparse", "urlunparse", "urljoin", "urldefrag",
48995
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
8 "urlsplit", "urlunsplit", "parse_qs", "parse_qsl"]
16889
0feb63490093 final round of __all__ lists (I hope) - skipped urllib2 because Moshe may be
Skip Montanaro <skip@pobox.com>
parents: 15874
diff changeset
9
1875
8c9cc054c5ed New tty/pty modules by Steen; new urlparser.
Guido van Rossum <guido@python.org>
parents:
diff changeset
10 # A classification of schemes ('' means apply by default)
32004
e4f52e2e3800 Revert last change.
Raymond Hettinger <python@rcn.com>
parents: 32003
diff changeset
11 uses_relative = ['ftp', 'http', 'gopher', 'nntp', 'imap',
36339
76aba3fa4e2f Bug #1407902: Added support for sftp:// URIs to urlparse.
Georg Brandl <georg@python.org>
parents: 35289
diff changeset
12 'wais', 'file', 'https', 'shttp', 'mms',
76aba3fa4e2f Bug #1407902: Added support for sftp:// URIs to urlparse.
Georg Brandl <georg@python.org>
parents: 35289
diff changeset
13 'prospero', 'rtsp', 'rtspu', '', 'sftp']
32004
e4f52e2e3800 Revert last change.
Raymond Hettinger <python@rcn.com>
parents: 32003
diff changeset
14 uses_netloc = ['ftp', 'http', 'gopher', 'nntp', 'telnet',
36339
76aba3fa4e2f Bug #1407902: Added support for sftp:// URIs to urlparse.
Georg Brandl <georg@python.org>
parents: 35289
diff changeset
15 'imap', 'wais', 'file', 'mms', 'https', 'shttp',
76aba3fa4e2f Bug #1407902: Added support for sftp:// URIs to urlparse.
Georg Brandl <georg@python.org>
parents: 35289
diff changeset
16 'snews', 'prospero', 'rtsp', 'rtspu', 'rsync', '',
52476
2ff89f0f1d19 Fix for bugs: Issue4675 and Issue4962.
Senthil Kumaran <orsenthil@gmail.com>
parents: 48995
diff changeset
17 'svn', 'svn+ssh', 'sftp','nfs']
32004
e4f52e2e3800 Revert last change.
Raymond Hettinger <python@rcn.com>
parents: 32003
diff changeset
18 non_hierarchical = ['gopher', 'hdl', 'mailto', 'news',
37368
05914e8ded73 add support for the sips: scheme (identical to sip: except for scheme name)
Fred Drake <fdrake@acm.org>
parents: 36339
diff changeset
19 'telnet', 'wais', 'imap', 'snews', 'sip', 'sips']
32004
e4f52e2e3800 Revert last change.
Raymond Hettinger <python@rcn.com>
parents: 32003
diff changeset
20 uses_params = ['ftp', 'hdl', 'prospero', 'http', 'imap',
37368
05914e8ded73 add support for the sips: scheme (identical to sip: except for scheme name)
Fred Drake <fdrake@acm.org>
parents: 36339
diff changeset
21 'https', 'shttp', 'rtsp', 'rtspu', 'sip', 'sips',
36339
76aba3fa4e2f Bug #1407902: Added support for sftp:// URIs to urlparse.
Georg Brandl <georg@python.org>
parents: 35289
diff changeset
22 'mms', '', 'sftp']
32004
e4f52e2e3800 Revert last change.
Raymond Hettinger <python@rcn.com>
parents: 32003
diff changeset
23 uses_query = ['http', 'wais', 'imap', 'https', 'shttp', 'mms',
37368
05914e8ded73 add support for the sips: scheme (identical to sip: except for scheme name)
Fred Drake <fdrake@acm.org>
parents: 36339
diff changeset
24 'gopher', 'rtsp', 'rtspu', 'sip', 'sips', '']
32004
e4f52e2e3800 Revert last change.
Raymond Hettinger <python@rcn.com>
parents: 32003
diff changeset
25 uses_fragment = ['ftp', 'hdl', 'http', 'gopher', 'news',
36339
76aba3fa4e2f Bug #1407902: Added support for sftp:// URIs to urlparse.
Georg Brandl <georg@python.org>
parents: 35289
diff changeset
26 'nntp', 'wais', 'https', 'shttp', 'snews',
76aba3fa4e2f Bug #1407902: Added support for sftp:// URIs to urlparse.
Georg Brandl <georg@python.org>
parents: 35289
diff changeset
27 'file', 'prospero', '']
1875
8c9cc054c5ed New tty/pty modules by Steen; new urlparser.
Guido van Rossum <guido@python.org>
parents:
diff changeset
28
8c9cc054c5ed New tty/pty modules by Steen; new urlparser.
Guido van Rossum <guido@python.org>
parents:
diff changeset
29 # Characters valid in scheme names
15623
db69c092c906 Be explicit about scheme_chars -- string.letters is locale dependent
Guido van Rossum <guido@python.org>
parents: 12967
diff changeset
30 scheme_chars = ('abcdefghijklmnopqrstuvwxyz'
db69c092c906 Be explicit about scheme_chars -- string.letters is locale dependent
Guido van Rossum <guido@python.org>
parents: 12967
diff changeset
31 'ABCDEFGHIJKLMNOPQRSTUVWXYZ'
db69c092c906 Be explicit about scheme_chars -- string.letters is locale dependent
Guido van Rossum <guido@python.org>
parents: 12967
diff changeset
32 '0123456789'
db69c092c906 Be explicit about scheme_chars -- string.letters is locale dependent
Guido van Rossum <guido@python.org>
parents: 12967
diff changeset
33 '+-.')
1875
8c9cc054c5ed New tty/pty modules by Steen; new urlparser.
Guido van Rossum <guido@python.org>
parents:
diff changeset
34
5785
b247297be9a0 After some discussion with Jeremy and Fred, decided to limit the
Guido van Rossum <guido@python.org>
parents: 5776
diff changeset
35 MAX_CACHE_SIZE = 20
3652
728a39a3e0d5 optimizations due to Fred Drake; added urldefrag() function
Guido van Rossum <guido@python.org>
parents: 3641
diff changeset
36 _parse_cache = {}
728a39a3e0d5 optimizations due to Fred Drake; added urldefrag() function
Guido van Rossum <guido@python.org>
parents: 3641
diff changeset
37
728a39a3e0d5 optimizations due to Fred Drake; added urldefrag() function
Guido van Rossum <guido@python.org>
parents: 3641
diff changeset
38 def clear_cache():
15874
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
39 """Clear the parse cache."""
44644
79579fa8d752 Speed-up and simplify code urlparse's result objects.
Raymond Hettinger <python@rcn.com>
parents: 44510
diff changeset
40 _parse_cache.clear()
3652
728a39a3e0d5 optimizations due to Fred Drake; added urldefrag() function
Guido van Rossum <guido@python.org>
parents: 3641
diff changeset
41
728a39a3e0d5 optimizations due to Fred Drake; added urldefrag() function
Guido van Rossum <guido@python.org>
parents: 3641
diff changeset
42
44644
79579fa8d752 Speed-up and simplify code urlparse's result objects.
Raymond Hettinger <python@rcn.com>
parents: 44510
diff changeset
43 class ResultMixin(object):
79579fa8d752 Speed-up and simplify code urlparse's result objects.
Raymond Hettinger <python@rcn.com>
parents: 44510
diff changeset
44 """Shared methods for the parsed result objects."""
37393
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
45
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
46 @property
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
47 def username(self):
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
48 netloc = self.netloc
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
49 if "@" in netloc:
44483
2e2fa291914c Patch #1698 by Senthil: allow '@' in username when parsed by urlparse.py.
Guido van Rossum <guido@python.org>
parents: 44284
diff changeset
50 userinfo = netloc.rsplit("@", 1)[0]
37393
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
51 if ":" in userinfo:
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
52 userinfo = userinfo.split(":", 1)[0]
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
53 return userinfo
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
54 return None
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
55
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
56 @property
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
57 def password(self):
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
58 netloc = self.netloc
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
59 if "@" in netloc:
44483
2e2fa291914c Patch #1698 by Senthil: allow '@' in username when parsed by urlparse.py.
Guido van Rossum <guido@python.org>
parents: 44284
diff changeset
60 userinfo = netloc.rsplit("@", 1)[0]
37393
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
61 if ":" in userinfo:
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
62 return userinfo.split(":", 1)[1]
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
63 return None
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
64
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
65 @property
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
66 def hostname(self):
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
67 netloc = self.netloc
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
68 if "@" in netloc:
44483
2e2fa291914c Patch #1698 by Senthil: allow '@' in username when parsed by urlparse.py.
Guido van Rossum <guido@python.org>
parents: 44284
diff changeset
69 netloc = netloc.rsplit("@", 1)[1]
37393
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
70 if ":" in netloc:
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
71 netloc = netloc.split(":", 1)[0]
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
72 return netloc.lower() or None
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
73
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
74 @property
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
75 def port(self):
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
76 netloc = self.netloc
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
77 if "@" in netloc:
44483
2e2fa291914c Patch #1698 by Senthil: allow '@' in username when parsed by urlparse.py.
Guido van Rossum <guido@python.org>
parents: 44284
diff changeset
78 netloc = netloc.rsplit("@", 1)[1]
37393
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
79 if ":" in netloc:
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
80 port = netloc.split(":", 1)[1]
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
81 return int(port, 10)
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
82 return None
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
83
44644
79579fa8d752 Speed-up and simplify code urlparse's result objects.
Raymond Hettinger <python@rcn.com>
parents: 44510
diff changeset
84 from collections import namedtuple
37393
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
85
44644
79579fa8d752 Speed-up and simplify code urlparse's result objects.
Raymond Hettinger <python@rcn.com>
parents: 44510
diff changeset
86 class SplitResult(namedtuple('SplitResult', 'scheme netloc path query fragment'), ResultMixin):
37393
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
87
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
88 __slots__ = ()
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
89
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
90 def geturl(self):
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
91 return urlunsplit(self)
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
92
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
93
44644
79579fa8d752 Speed-up and simplify code urlparse's result objects.
Raymond Hettinger <python@rcn.com>
parents: 44510
diff changeset
94 class ParseResult(namedtuple('ParseResult', 'scheme netloc path params query fragment'), ResultMixin):
37393
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
95
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
96 __slots__ = ()
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
97
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
98 def geturl(self):
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
99 return urlunparse(self)
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
100
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
101
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
102 def urlparse(url, scheme='', allow_fragments=True):
15874
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
103 """Parse a URL into 6 components:
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
104 <scheme>://<netloc>/<path>;<params>?<query>#<fragment>
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
105 Return a 6-tuple: (scheme, netloc, path, params, query, fragment).
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
106 Note that we don't break the components up in smaller bits
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
107 (e.g. netloc is a single string) and we don't expand % escapes."""
20995
174ccb035b3d Fix parsing of parameters from a URL; urlparse() did not check that it only
Fred Drake <fdrake@acm.org>
parents: 19209
diff changeset
108 tuple = urlsplit(url, scheme, allow_fragments)
174ccb035b3d Fix parsing of parameters from a URL; urlparse() did not check that it only
Fred Drake <fdrake@acm.org>
parents: 19209
diff changeset
109 scheme, netloc, url, query, fragment = tuple
174ccb035b3d Fix parsing of parameters from a URL; urlparse() did not check that it only
Fred Drake <fdrake@acm.org>
parents: 19209
diff changeset
110 if scheme in uses_params and ';' in url:
174ccb035b3d Fix parsing of parameters from a URL; urlparse() did not check that it only
Fred Drake <fdrake@acm.org>
parents: 19209
diff changeset
111 url, params = _splitparams(url)
174ccb035b3d Fix parsing of parameters from a URL; urlparse() did not check that it only
Fred Drake <fdrake@acm.org>
parents: 19209
diff changeset
112 else:
174ccb035b3d Fix parsing of parameters from a URL; urlparse() did not check that it only
Fred Drake <fdrake@acm.org>
parents: 19209
diff changeset
113 params = ''
37393
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
114 return ParseResult(scheme, netloc, url, params, query, fragment)
20995
174ccb035b3d Fix parsing of parameters from a URL; urlparse() did not check that it only
Fred Drake <fdrake@acm.org>
parents: 19209
diff changeset
115
174ccb035b3d Fix parsing of parameters from a URL; urlparse() did not check that it only
Fred Drake <fdrake@acm.org>
parents: 19209
diff changeset
116 def _splitparams(url):
174ccb035b3d Fix parsing of parameters from a URL; urlparse() did not check that it only
Fred Drake <fdrake@acm.org>
parents: 19209
diff changeset
117 if '/' in url:
174ccb035b3d Fix parsing of parameters from a URL; urlparse() did not check that it only
Fred Drake <fdrake@acm.org>
parents: 19209
diff changeset
118 i = url.find(';', url.rfind('/'))
174ccb035b3d Fix parsing of parameters from a URL; urlparse() did not check that it only
Fred Drake <fdrake@acm.org>
parents: 19209
diff changeset
119 if i < 0:
174ccb035b3d Fix parsing of parameters from a URL; urlparse() did not check that it only
Fred Drake <fdrake@acm.org>
parents: 19209
diff changeset
120 return url, ''
174ccb035b3d Fix parsing of parameters from a URL; urlparse() did not check that it only
Fred Drake <fdrake@acm.org>
parents: 19209
diff changeset
121 else:
174ccb035b3d Fix parsing of parameters from a URL; urlparse() did not check that it only
Fred Drake <fdrake@acm.org>
parents: 19209
diff changeset
122 i = url.find(';')
174ccb035b3d Fix parsing of parameters from a URL; urlparse() did not check that it only
Fred Drake <fdrake@acm.org>
parents: 19209
diff changeset
123 return url[:i], url[i+1:]
174ccb035b3d Fix parsing of parameters from a URL; urlparse() did not check that it only
Fred Drake <fdrake@acm.org>
parents: 19209
diff changeset
124
34378
a0bc4419663c Patch #712317: In URLs such as http://www.example.com?query=spam, treat '?' as
Johannes Gijsbers <jlg@dds.nl>
parents: 34293
diff changeset
125 def _splitnetloc(url, start=0):
44510
65fe6e56d4f8 Patch #1637: fix urlparse for URLs like 'http://x.com?arg=/foo'.
Guido van Rossum <guido@python.org>
parents: 44483
diff changeset
126 delim = len(url) # position of end of domain part of url, default is end
65fe6e56d4f8 Patch #1637: fix urlparse for URLs like 'http://x.com?arg=/foo'.
Guido van Rossum <guido@python.org>
parents: 44483
diff changeset
127 for c in '/?#': # look for delimiters; the order is NOT important
65fe6e56d4f8 Patch #1637: fix urlparse for URLs like 'http://x.com?arg=/foo'.
Guido van Rossum <guido@python.org>
parents: 44483
diff changeset
128 wdelim = url.find(c, start) # find first of this delim
65fe6e56d4f8 Patch #1637: fix urlparse for URLs like 'http://x.com?arg=/foo'.
Guido van Rossum <guido@python.org>
parents: 44483
diff changeset
129 if wdelim >= 0: # if found
65fe6e56d4f8 Patch #1637: fix urlparse for URLs like 'http://x.com?arg=/foo'.
Guido van Rossum <guido@python.org>
parents: 44483
diff changeset
130 delim = min(delim, wdelim) # use earliest delim position
65fe6e56d4f8 Patch #1637: fix urlparse for URLs like 'http://x.com?arg=/foo'.
Guido van Rossum <guido@python.org>
parents: 44483
diff changeset
131 return url[start:delim], url[delim:] # return (domain, rest)
34378
a0bc4419663c Patch #712317: In URLs such as http://www.example.com?query=spam, treat '?' as
Johannes Gijsbers <jlg@dds.nl>
parents: 34293
diff changeset
132
37393
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
133 def urlsplit(url, scheme='', allow_fragments=True):
20995
174ccb035b3d Fix parsing of parameters from a URL; urlparse() did not check that it only
Fred Drake <fdrake@acm.org>
parents: 19209
diff changeset
134 """Parse a URL into 5 components:
174ccb035b3d Fix parsing of parameters from a URL; urlparse() did not check that it only
Fred Drake <fdrake@acm.org>
parents: 19209
diff changeset
135 <scheme>://<netloc>/<path>?<query>#<fragment>
174ccb035b3d Fix parsing of parameters from a URL; urlparse() did not check that it only
Fred Drake <fdrake@acm.org>
parents: 19209
diff changeset
136 Return a 5-tuple: (scheme, netloc, path, query, fragment).
174ccb035b3d Fix parsing of parameters from a URL; urlparse() did not check that it only
Fred Drake <fdrake@acm.org>
parents: 19209
diff changeset
137 Note that we don't break the components up in smaller bits
174ccb035b3d Fix parsing of parameters from a URL; urlparse() did not check that it only
Fred Drake <fdrake@acm.org>
parents: 19209
diff changeset
138 (e.g. netloc is a single string) and we don't expand % escapes."""
37393
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
139 allow_fragments = bool(allow_fragments)
44284
6ae94a2ba5df Backport r59480.
Alexandre Vassalotti <alexandre@peadrop.com>
parents: 37393
diff changeset
140 key = url, scheme, allow_fragments, type(url), type(scheme)
15874
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
141 cached = _parse_cache.get(key, None)
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
142 if cached:
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
143 return cached
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
144 if len(_parse_cache) >= MAX_CACHE_SIZE: # avoid runaway growth
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
145 clear_cache()
20995
174ccb035b3d Fix parsing of parameters from a URL; urlparse() did not check that it only
Fred Drake <fdrake@acm.org>
parents: 19209
diff changeset
146 netloc = query = fragment = ''
15874
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
147 i = url.find(':')
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
148 if i > 0:
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
149 if url[:i] == 'http': # optimize the common case
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
150 scheme = url[:i].lower()
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
151 url = url[i+1:]
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
152 if url[:2] == '//':
34378
a0bc4419663c Patch #712317: In URLs such as http://www.example.com?query=spam, treat '?' as
Johannes Gijsbers <jlg@dds.nl>
parents: 34293
diff changeset
153 netloc, url = _splitnetloc(url, 2)
20995
174ccb035b3d Fix parsing of parameters from a URL; urlparse() did not check that it only
Fred Drake <fdrake@acm.org>
parents: 19209
diff changeset
154 if allow_fragments and '#' in url:
174ccb035b3d Fix parsing of parameters from a URL; urlparse() did not check that it only
Fred Drake <fdrake@acm.org>
parents: 19209
diff changeset
155 url, fragment = url.split('#', 1)
174ccb035b3d Fix parsing of parameters from a URL; urlparse() did not check that it only
Fred Drake <fdrake@acm.org>
parents: 19209
diff changeset
156 if '?' in url:
174ccb035b3d Fix parsing of parameters from a URL; urlparse() did not check that it only
Fred Drake <fdrake@acm.org>
parents: 19209
diff changeset
157 url, query = url.split('?', 1)
37393
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
158 v = SplitResult(scheme, netloc, url, query, fragment)
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
159 _parse_cache[key] = v
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
160 return v
15874
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
161 for c in url[:i]:
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
162 if c not in scheme_chars:
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
163 break
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
164 else:
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
165 scheme, url = url[:i].lower(), url[i+1:]
34378
a0bc4419663c Patch #712317: In URLs such as http://www.example.com?query=spam, treat '?' as
Johannes Gijsbers <jlg@dds.nl>
parents: 34293
diff changeset
166 if scheme in uses_netloc and url[:2] == '//':
a0bc4419663c Patch #712317: In URLs such as http://www.example.com?query=spam, treat '?' as
Johannes Gijsbers <jlg@dds.nl>
parents: 34293
diff changeset
167 netloc, url = _splitnetloc(url, 2)
20995
174ccb035b3d Fix parsing of parameters from a URL; urlparse() did not check that it only
Fred Drake <fdrake@acm.org>
parents: 19209
diff changeset
168 if allow_fragments and scheme in uses_fragment and '#' in url:
174ccb035b3d Fix parsing of parameters from a URL; urlparse() did not check that it only
Fred Drake <fdrake@acm.org>
parents: 19209
diff changeset
169 url, fragment = url.split('#', 1)
174ccb035b3d Fix parsing of parameters from a URL; urlparse() did not check that it only
Fred Drake <fdrake@acm.org>
parents: 19209
diff changeset
170 if scheme in uses_query and '?' in url:
174ccb035b3d Fix parsing of parameters from a URL; urlparse() did not check that it only
Fred Drake <fdrake@acm.org>
parents: 19209
diff changeset
171 url, query = url.split('?', 1)
37393
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
172 v = SplitResult(scheme, netloc, url, query, fragment)
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
173 _parse_cache[key] = v
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
174 return v
1875
8c9cc054c5ed New tty/pty modules by Steen; new urlparser.
Guido van Rossum <guido@python.org>
parents:
diff changeset
175
48140
cc43b998f5b5 Silence some SyntaxWarnings for tuple unpacking in a parameter list for
Brett Cannon <bcannon@gmail.com>
parents: 44847
diff changeset
176 def urlunparse(data):
15874
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
177 """Put a parsed URL back together again. This may result in a
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
178 slightly different, but equivalent URL, if the URL that was parsed
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
179 originally had redundant delimiters, e.g. a ? with an empty query
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
180 (the draft states that these are equivalent)."""
48140
cc43b998f5b5 Silence some SyntaxWarnings for tuple unpacking in a parameter list for
Brett Cannon <bcannon@gmail.com>
parents: 44847
diff changeset
181 scheme, netloc, url, params, query, fragment = data
20995
174ccb035b3d Fix parsing of parameters from a URL; urlparse() did not check that it only
Fred Drake <fdrake@acm.org>
parents: 19209
diff changeset
182 if params:
174ccb035b3d Fix parsing of parameters from a URL; urlparse() did not check that it only
Fred Drake <fdrake@acm.org>
parents: 19209
diff changeset
183 url = "%s;%s" % (url, params)
174ccb035b3d Fix parsing of parameters from a URL; urlparse() did not check that it only
Fred Drake <fdrake@acm.org>
parents: 19209
diff changeset
184 return urlunsplit((scheme, netloc, url, query, fragment))
174ccb035b3d Fix parsing of parameters from a URL; urlparse() did not check that it only
Fred Drake <fdrake@acm.org>
parents: 19209
diff changeset
185
48140
cc43b998f5b5 Silence some SyntaxWarnings for tuple unpacking in a parameter list for
Brett Cannon <bcannon@gmail.com>
parents: 44847
diff changeset
186 def urlunsplit(data):
cc43b998f5b5 Silence some SyntaxWarnings for tuple unpacking in a parameter list for
Brett Cannon <bcannon@gmail.com>
parents: 44847
diff changeset
187 scheme, netloc, url, query, fragment = data
25787
bf384111b3b3 Backporting urlparse.py:1.34, test_urlparse.py:1.8:
Guido van Rossum <guido@python.org>
parents: 25425
diff changeset
188 if netloc or (scheme and scheme in uses_netloc and url[:2] != '//'):
15874
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
189 if url and url[:1] != '/': url = '/' + url
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
190 url = '//' + (netloc or '') + url
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
191 if scheme:
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
192 url = scheme + ':' + url
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
193 if query:
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
194 url = url + '?' + query
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
195 if fragment:
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
196 url = url + '#' + fragment
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
197 return url
1875
8c9cc054c5ed New tty/pty modules by Steen; new urlparser.
Guido van Rossum <guido@python.org>
parents:
diff changeset
198
37393
a6bce7aebe05 Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results
Fred Drake <fdrake@acm.org>
parents: 37368
diff changeset
199 def urljoin(base, url, allow_fragments=True):
15874
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
200 """Join a base URL and a possibly relative URL to form an absolute
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
201 interpretation of the latter."""
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
202 if not base:
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
203 return url
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
204 if not url:
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
205 return base
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
206 bscheme, bnetloc, bpath, bparams, bquery, bfragment = \
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
207 urlparse(base, '', allow_fragments)
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
208 scheme, netloc, path, params, query, fragment = \
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
209 urlparse(url, bscheme, allow_fragments)
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
210 if scheme != bscheme or scheme not in uses_relative:
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
211 return url
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
212 if scheme in uses_netloc:
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
213 if netloc:
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
214 return urlunparse((scheme, netloc, path,
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
215 params, query, fragment))
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
216 netloc = bnetloc
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
217 if path[:1] == '/':
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
218 return urlunparse((scheme, netloc, path,
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
219 params, query, fragment))
48233
af7388ad3164 Issue 1432. Fixes a bug caused because of the evolution
Facundo Batista <facundobatista@gmail.com>
parents: 48140
diff changeset
220 if not path:
af7388ad3164 Issue 1432. Fixes a bug caused because of the evolution
Facundo Batista <facundobatista@gmail.com>
parents: 48140
diff changeset
221 path = bpath
af7388ad3164 Issue 1432. Fixes a bug caused because of the evolution
Facundo Batista <facundobatista@gmail.com>
parents: 48140
diff changeset
222 if not params:
af7388ad3164 Issue 1432. Fixes a bug caused because of the evolution
Facundo Batista <facundobatista@gmail.com>
parents: 48140
diff changeset
223 params = bparams
af7388ad3164 Issue 1432. Fixes a bug caused because of the evolution
Facundo Batista <facundobatista@gmail.com>
parents: 48140
diff changeset
224 else:
af7388ad3164 Issue 1432. Fixes a bug caused because of the evolution
Facundo Batista <facundobatista@gmail.com>
parents: 48140
diff changeset
225 path = path[:-1]
af7388ad3164 Issue 1432. Fixes a bug caused because of the evolution
Facundo Batista <facundobatista@gmail.com>
parents: 48140
diff changeset
226 return urlunparse((scheme, netloc, path,
af7388ad3164 Issue 1432. Fixes a bug caused because of the evolution
Facundo Batista <facundobatista@gmail.com>
parents: 48140
diff changeset
227 params, query, fragment))
af7388ad3164 Issue 1432. Fixes a bug caused because of the evolution
Facundo Batista <facundobatista@gmail.com>
parents: 48140
diff changeset
228 if not query:
af7388ad3164 Issue 1432. Fixes a bug caused because of the evolution
Facundo Batista <facundobatista@gmail.com>
parents: 48140
diff changeset
229 query = bquery
af7388ad3164 Issue 1432. Fixes a bug caused because of the evolution
Facundo Batista <facundobatista@gmail.com>
parents: 48140
diff changeset
230 return urlunparse((scheme, netloc, path,
af7388ad3164 Issue 1432. Fixes a bug caused because of the evolution
Facundo Batista <facundobatista@gmail.com>
parents: 48140
diff changeset
231 params, query, fragment))
15874
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
232 segments = bpath.split('/')[:-1] + path.split('/')
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
233 # XXX The stuff below is bogus in various ways...
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
234 if segments[-1] == '.':
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
235 segments[-1] = ''
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
236 while '.' in segments:
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
237 segments.remove('.')
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
238 while 1:
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
239 i = 1
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
240 n = len(segments) - 1
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
241 while i < n:
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
242 if (segments[i] == '..'
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
243 and segments[i-1] not in ('', '..')):
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
244 del segments[i-1:i+1]
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
245 break
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
246 i = i+1
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
247 else:
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
248 break
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
249 if segments == ['', '..']:
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
250 segments[-1] = ''
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
251 elif len(segments) >= 2 and segments[-1] == '..':
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
252 segments[-2:] = ['']
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
253 return urlunparse((scheme, netloc, '/'.join(segments),
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
254 params, query, fragment))
1875
8c9cc054c5ed New tty/pty modules by Steen; new urlparser.
Guido van Rossum <guido@python.org>
parents:
diff changeset
255
3652
728a39a3e0d5 optimizations due to Fred Drake; added urldefrag() function
Guido van Rossum <guido@python.org>
parents: 3641
diff changeset
256 def urldefrag(url):
15874
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
257 """Removes any existing fragment from URL.
3652
728a39a3e0d5 optimizations due to Fred Drake; added urldefrag() function
Guido van Rossum <guido@python.org>
parents: 3641
diff changeset
258
15874
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
259 Returns a tuple of the defragmented URL and the fragment. If
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
260 the URL contained no fragments, the second element is the
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
261 empty string.
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
262 """
20995
174ccb035b3d Fix parsing of parameters from a URL; urlparse() did not check that it only
Fred Drake <fdrake@acm.org>
parents: 19209
diff changeset
263 if '#' in url:
174ccb035b3d Fix parsing of parameters from a URL; urlparse() did not check that it only
Fred Drake <fdrake@acm.org>
parents: 19209
diff changeset
264 s, n, p, a, q, frag = urlparse(url)
174ccb035b3d Fix parsing of parameters from a URL; urlparse() did not check that it only
Fred Drake <fdrake@acm.org>
parents: 19209
diff changeset
265 defrag = urlunparse((s, n, p, a, q, ''))
174ccb035b3d Fix parsing of parameters from a URL; urlparse() did not check that it only
Fred Drake <fdrake@acm.org>
parents: 19209
diff changeset
266 return defrag, frag
174ccb035b3d Fix parsing of parameters from a URL; urlparse() did not check that it only
Fred Drake <fdrake@acm.org>
parents: 19209
diff changeset
267 else:
174ccb035b3d Fix parsing of parameters from a URL; urlparse() did not check that it only
Fred Drake <fdrake@acm.org>
parents: 19209
diff changeset
268 return url, ''
3652
728a39a3e0d5 optimizations due to Fred Drake; added urldefrag() function
Guido van Rossum <guido@python.org>
parents: 3641
diff changeset
269
48995
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
270 # unquote method for parse_qs and parse_qsl
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
271 # Cannot use directly from urllib as it would create circular reference.
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
272 # urllib uses urlparse methods ( urljoin)
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
273
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
274 _hextochr = dict(('%02x' % i, chr(i)) for i in range(256))
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
275 _hextochr.update(('%02X' % i, chr(i)) for i in range(256))
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
276
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
277 def unquote(s):
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
278 """unquote('abc%20def') -> 'abc def'."""
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
279 res = s.split('%')
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
280 for i in xrange(1, len(res)):
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
281 item = res[i]
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
282 try:
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
283 res[i] = _hextochr[item[:2]] + item[2:]
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
284 except KeyError:
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
285 res[i] = '%' + item
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
286 except UnicodeDecodeError:
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
287 res[i] = unichr(int(item[:2], 16)) + item[2:]
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
288 return "".join(res)
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
289
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
290 def parse_qs(qs, keep_blank_values=0, strict_parsing=0):
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
291 """Parse a query given as a string argument.
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
292
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
293 Arguments:
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
294
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
295 qs: URL-encoded query string to be parsed
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
296
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
297 keep_blank_values: flag indicating whether blank values in
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
298 URL encoded queries should be treated as blank strings.
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
299 A true value indicates that blanks should be retained as
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
300 blank strings. The default false value indicates that
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
301 blank values are to be ignored and treated as if they were
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
302 not included.
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
303
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
304 strict_parsing: flag indicating what to do with parsing errors.
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
305 If false (the default), errors are silently ignored.
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
306 If true, errors raise a ValueError exception.
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
307 """
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
308 dict = {}
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
309 for name, value in parse_qsl(qs, keep_blank_values, strict_parsing):
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
310 if name in dict:
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
311 dict[name].append(value)
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
312 else:
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
313 dict[name] = [value]
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
314 return dict
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
315
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
316 def parse_qsl(qs, keep_blank_values=0, strict_parsing=0):
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
317 """Parse a query given as a string argument.
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
318
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
319 Arguments:
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
320
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
321 qs: URL-encoded query string to be parsed
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
322
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
323 keep_blank_values: flag indicating whether blank values in
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
324 URL encoded queries should be treated as blank strings. A
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
325 true value indicates that blanks should be retained as blank
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
326 strings. The default false value indicates that blank values
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
327 are to be ignored and treated as if they were not included.
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
328
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
329 strict_parsing: flag indicating what to do with parsing errors. If
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
330 false (the default), errors are silently ignored. If true,
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
331 errors raise a ValueError exception.
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
332
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
333 Returns a list, as G-d intended.
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
334 """
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
335 pairs = [s2 for s1 in qs.split('&') for s2 in s1.split(';')]
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
336 r = []
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
337 for name_value in pairs:
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
338 if not name_value and not strict_parsing:
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
339 continue
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
340 nv = name_value.split('=', 1)
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
341 if len(nv) != 2:
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
342 if strict_parsing:
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
343 raise ValueError, "bad query field: %r" % (name_value,)
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
344 # Handle case of a control-name with no equal sign
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
345 if keep_blank_values:
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
346 nv.append('')
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
347 else:
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
348 continue
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
349 if len(nv[1]) or keep_blank_values:
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
350 name = unquote(nv[0].replace('+', ' '))
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
351 value = unquote(nv[1].replace('+', ' '))
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
352 r.append((name, value))
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
353
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
354 return r
2b855a947f0f Issue 600362: Relocated parse_qs() and parse_qsl(), from the cgi module
Facundo Batista <facundobatista@gmail.com>
parents: 48233
diff changeset
355
3652
728a39a3e0d5 optimizations due to Fred Drake; added urldefrag() function
Guido van Rossum <guido@python.org>
parents: 3641
diff changeset
356
1875
8c9cc054c5ed New tty/pty modules by Steen; new urlparser.
Guido van Rossum <guido@python.org>
parents:
diff changeset
357 test_input = """
8c9cc054c5ed New tty/pty modules by Steen; new urlparser.
Guido van Rossum <guido@python.org>
parents:
diff changeset
358 http://a/b/c/d
8c9cc054c5ed New tty/pty modules by Steen; new urlparser.
Guido van Rossum <guido@python.org>
parents:
diff changeset
359
8c9cc054c5ed New tty/pty modules by Steen; new urlparser.
Guido van Rossum <guido@python.org>
parents:
diff changeset
360 g:h = <URL:g:h>
8c9cc054c5ed New tty/pty modules by Steen; new urlparser.
Guido van Rossum <guido@python.org>
parents:
diff changeset
361 http:g = <URL:http://a/b/c/g>
8c9cc054c5ed New tty/pty modules by Steen; new urlparser.
Guido van Rossum <guido@python.org>
parents:
diff changeset
362 http: = <URL:http://a/b/c/d>
8c9cc054c5ed New tty/pty modules by Steen; new urlparser.
Guido van Rossum <guido@python.org>
parents:
diff changeset
363 g = <URL:http://a/b/c/g>
8c9cc054c5ed New tty/pty modules by Steen; new urlparser.
Guido van Rossum <guido@python.org>
parents:
diff changeset
364 ./g = <URL:http://a/b/c/g>
8c9cc054c5ed New tty/pty modules by Steen; new urlparser.
Guido van Rossum <guido@python.org>
parents:
diff changeset
365 g/ = <URL:http://a/b/c/g/>
8c9cc054c5ed New tty/pty modules by Steen; new urlparser.
Guido van Rossum <guido@python.org>
parents:
diff changeset
366 /g = <URL:http://a/g>
8c9cc054c5ed New tty/pty modules by Steen; new urlparser.
Guido van Rossum <guido@python.org>
parents:
diff changeset
367 //g = <URL:http://g>
8c9cc054c5ed New tty/pty modules by Steen; new urlparser.
Guido van Rossum <guido@python.org>
parents:
diff changeset
368 ?y = <URL:http://a/b/c/d?y>
8c9cc054c5ed New tty/pty modules by Steen; new urlparser.
Guido van Rossum <guido@python.org>
parents:
diff changeset
369 g?y = <URL:http://a/b/c/g?y>
8c9cc054c5ed New tty/pty modules by Steen; new urlparser.
Guido van Rossum <guido@python.org>
parents:
diff changeset
370 g?y/./x = <URL:http://a/b/c/g?y/./x>
8c9cc054c5ed New tty/pty modules by Steen; new urlparser.
Guido van Rossum <guido@python.org>
parents:
diff changeset
371 . = <URL:http://a/b/c/>
8c9cc054c5ed New tty/pty modules by Steen; new urlparser.
Guido van Rossum <guido@python.org>
parents:
diff changeset
372 ./ = <URL:http://a/b/c/>
8c9cc054c5ed New tty/pty modules by Steen; new urlparser.
Guido van Rossum <guido@python.org>
parents:
diff changeset
373 .. = <URL:http://a/b/>
8c9cc054c5ed New tty/pty modules by Steen; new urlparser.
Guido van Rossum <guido@python.org>
parents:
diff changeset
374 ../ = <URL:http://a/b/>
8c9cc054c5ed New tty/pty modules by Steen; new urlparser.
Guido van Rossum <guido@python.org>
parents:
diff changeset
375 ../g = <URL:http://a/b/g>
8c9cc054c5ed New tty/pty modules by Steen; new urlparser.
Guido van Rossum <guido@python.org>
parents:
diff changeset
376 ../.. = <URL:http://a/>
8c9cc054c5ed New tty/pty modules by Steen; new urlparser.
Guido van Rossum <guido@python.org>
parents:
diff changeset
377 ../../g = <URL:http://a/g>
8c9cc054c5ed New tty/pty modules by Steen; new urlparser.
Guido van Rossum <guido@python.org>
parents:
diff changeset
378 ../../../g = <URL:http://a/../g>
8c9cc054c5ed New tty/pty modules by Steen; new urlparser.
Guido van Rossum <guido@python.org>
parents:
diff changeset
379 ./../g = <URL:http://a/b/g>
8c9cc054c5ed New tty/pty modules by Steen; new urlparser.
Guido van Rossum <guido@python.org>
parents:
diff changeset
380 ./g/. = <URL:http://a/b/c/g/>
8c9cc054c5ed New tty/pty modules by Steen; new urlparser.
Guido van Rossum <guido@python.org>
parents:
diff changeset
381 /./g = <URL:http://a/./g>
8c9cc054c5ed New tty/pty modules by Steen; new urlparser.
Guido van Rossum <guido@python.org>
parents:
diff changeset
382 g/./h = <URL:http://a/b/c/g/h>
8c9cc054c5ed New tty/pty modules by Steen; new urlparser.
Guido van Rossum <guido@python.org>
parents:
diff changeset
383 g/../h = <URL:http://a/b/c/h>
8c9cc054c5ed New tty/pty modules by Steen; new urlparser.
Guido van Rossum <guido@python.org>
parents:
diff changeset
384 http:g = <URL:http://a/b/c/g>
8c9cc054c5ed New tty/pty modules by Steen; new urlparser.
Guido van Rossum <guido@python.org>
parents:
diff changeset
385 http: = <URL:http://a/b/c/d>
9793
737d05a9bb5e Fixed bug in the common-case code for HTTP URLs; it would lose the query,
Andrew M. Kuchling <amk@amk.ca>
parents: 9665
diff changeset
386 http:?y = <URL:http://a/b/c/d?y>
737d05a9bb5e Fixed bug in the common-case code for HTTP URLs; it would lose the query,
Andrew M. Kuchling <amk@amk.ca>
parents: 9665
diff changeset
387 http:g?y = <URL:http://a/b/c/g?y>
737d05a9bb5e Fixed bug in the common-case code for HTTP URLs; it would lose the query,
Andrew M. Kuchling <amk@amk.ca>
parents: 9665
diff changeset
388 http:g?y/./x = <URL:http://a/b/c/g?y/./x>
1875
8c9cc054c5ed New tty/pty modules by Steen; new urlparser.
Guido van Rossum <guido@python.org>
parents:
diff changeset
389 """
8c9cc054c5ed New tty/pty modules by Steen; new urlparser.
Guido van Rossum <guido@python.org>
parents:
diff changeset
390
8c9cc054c5ed New tty/pty modules by Steen; new urlparser.
Guido van Rossum <guido@python.org>
parents:
diff changeset
391 def test():
15874
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
392 import sys
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
393 base = ''
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
394 if sys.argv[1:]:
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
395 fn = sys.argv[1]
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
396 if fn == '-':
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
397 fp = sys.stdin
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
398 else:
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
399 fp = open(fn)
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
400 else:
34293
cd517701fd93 Use cStringIO where available.
Raymond Hettinger <python@rcn.com>
parents: 32435
diff changeset
401 try:
cd517701fd93 Use cStringIO where available.
Raymond Hettinger <python@rcn.com>
parents: 32435
diff changeset
402 from cStringIO import StringIO
cd517701fd93 Use cStringIO where available.
Raymond Hettinger <python@rcn.com>
parents: 32435
diff changeset
403 except ImportError:
cd517701fd93 Use cStringIO where available.
Raymond Hettinger <python@rcn.com>
parents: 32435
diff changeset
404 from StringIO import StringIO
cd517701fd93 Use cStringIO where available.
Raymond Hettinger <python@rcn.com>
parents: 32435
diff changeset
405 fp = StringIO(test_input)
44847
a6b190b0cf06 #1726198: replace while 1: fp.readline() with file iteration.
Georg Brandl <georg@python.org>
parents: 44644
diff changeset
406 for line in fp:
15874
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
407 words = line.split()
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
408 if not words:
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
409 continue
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
410 url = words[0]
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
411 parts = urlparse(url)
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
412 print '%-10s : %s' % (url, parts)
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
413 abs = urljoin(base, url)
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
414 if not base:
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
415 base = abs
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
416 wrapped = '<URL:%s>' % abs
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
417 print '%-10s = %s' % (url, wrapped)
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
418 if len(words) == 3 and words[1] == '=':
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
419 if wrapped != words[2]:
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
420 print 'EXPECTED', words[2], '!!!!!!!!!!'
1875
8c9cc054c5ed New tty/pty modules by Steen; new urlparser.
Guido van Rossum <guido@python.org>
parents:
diff changeset
421
8c9cc054c5ed New tty/pty modules by Steen; new urlparser.
Guido van Rossum <guido@python.org>
parents:
diff changeset
422 if __name__ == '__main__':
15874
4c01ee0a8073 Whitespace normalization. Top level of Lib now fixed-point for reindent.py!
Tim Peters <tim.peters@gmail.com>
parents: 15741
diff changeset
423 test()