cpython-lru_cache-weakref: Lib/urllib.py annotate

annotate Lib/urllib.py @ 4566:6403c94de50d legacy-trunk

Added safeguard against failure in __del__.

author	Guido van Rossum <guido@python.org>
date	Wed, 27 Nov 1996 19:39:58 +0000
parents	df9d1ebcf7f3
children	e97528482bc6

rev	line source
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	1 # Open an arbitrary URL
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	2 #
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	3 # See the following document for a tentative description of URLs:
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	4 # Uniform Resource Locators Tim Berners-Lee
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	5 # INTERNET DRAFT CERN
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	6 # IETF URL Working Group 14 July 1993
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	7 # draft-ietf-uri-url-01.txt
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	8 #
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	9 # The object returned by URLopener().open(file) will differ per
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	10 # protocol. All you know is that is has methods read(), readline(),
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	11 # readlines(), fileno(), close() and info(). The read*(), fileno()
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	12 # and close() methods work like those of open files.
2899 65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	13 # The info() method returns an mimetools.Message object which can be
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	14 # used to query various info about the object, if available.
2899 65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	15 # (mimetools.Message objects are queried with the getheader() method.)
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	16
1701 f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	17 import string
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	18 import socket
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	19 import regex
3298 9a4081440783 Fixed local file access for macintosh Jack Jansen <jack.jansen@cwi.nl> parents: 3244 diff changeset	20 import os
4558 df9d1ebcf7f3 When re-raising an exception raised by a module used internally as Guido van Rossum <guido@python.org> parents: 4532 diff changeset	21 import sys
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	22
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	23
4257 239b10081176 One fix by sjoerd and one suggested by him. Bumped __version__ to 1.5. Guido van Rossum <guido@python.org> parents: 4153 diff changeset	24 __version__ = '1.5'
2779 af7d0fea3fb2 add User-agent hdr; read and close the file upon http error Guido van Rossum <guido@python.org> parents: 2047 diff changeset	25
3298 9a4081440783 Fixed local file access for macintosh Jack Jansen <jack.jansen@cwi.nl> parents: 3244 diff changeset	26 # Helper for non-unix systems
9a4081440783 Fixed local file access for macintosh Jack Jansen <jack.jansen@cwi.nl> parents: 3244 diff changeset	27 if os.name == 'mac':
3471 aa8adfdd3aa1 move mac url2path conversion to separate module Guido van Rossum <guido@python.org> parents: 3461 diff changeset	28 from macurl2path import url2pathname, pathname2url
3724 26442b5f9cfd add nturl2path Guido van Rossum <guido@python.org> parents: 3687 diff changeset	29 elif os.name == 'nt':
26442b5f9cfd add nturl2path Guido van Rossum <guido@python.org> parents: 3687 diff changeset	30 from nturl2path import url2pathname, pathname2url
3298 9a4081440783 Fixed local file access for macintosh Jack Jansen <jack.jansen@cwi.nl> parents: 3244 diff changeset	31 else:
3303 d4d5c9fa7859 Added pathname2url and url2pathname methods (only correct for unix and Jack Jansen <jack.jansen@cwi.nl> parents: 3298 diff changeset	32 def url2pathname(pathname):
3298 9a4081440783 Fixed local file access for macintosh Jack Jansen <jack.jansen@cwi.nl> parents: 3244 diff changeset	33 return pathname
3303 d4d5c9fa7859 Added pathname2url and url2pathname methods (only correct for unix and Jack Jansen <jack.jansen@cwi.nl> parents: 3298 diff changeset	34 def pathname2url(pathname):
d4d5c9fa7859 Added pathname2url and url2pathname methods (only correct for unix and Jack Jansen <jack.jansen@cwi.nl> parents: 3298 diff changeset	35 return pathname
2779 af7d0fea3fb2 add User-agent hdr; read and close the file upon http error Guido van Rossum <guido@python.org> parents: 2047 diff changeset	36
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	37 # This really consists of two pieces:
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	38 # (1) a class which handles opening of all sorts of URLs
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	39 # (plus assorted utilities etc.)
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	40 # (2) a set of functions for parsing URLs
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	41 # XXX Should these be separated out into different modules?
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	42
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	43
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	44 # Shortcut for basic usage
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	45 _urlopener = None
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	46 def urlopen(url):
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	47 global _urlopener
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	48 if not _urlopener:
2899 65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	49 _urlopener = FancyURLopener()
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	50 return _urlopener.open(url)
3660 058d81d1d56d Support optional filename argument for retrieve() and urlretrieve(), Guido van Rossum <guido@python.org> parents: 3471 diff changeset	51 def urlretrieve(url, filename=None):
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	52 global _urlopener
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	53 if not _urlopener:
2899 65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	54 _urlopener = FancyURLopener()
3660 058d81d1d56d Support optional filename argument for retrieve() and urlretrieve(), Guido van Rossum <guido@python.org> parents: 3471 diff changeset	55 if filename:
058d81d1d56d Support optional filename argument for retrieve() and urlretrieve(), Guido van Rossum <guido@python.org> parents: 3471 diff changeset	56 return _urlopener.retrieve(url, filename)
058d81d1d56d Support optional filename argument for retrieve() and urlretrieve(), Guido van Rossum <guido@python.org> parents: 3471 diff changeset	57 else:
058d81d1d56d Support optional filename argument for retrieve() and urlretrieve(), Guido van Rossum <guido@python.org> parents: 3471 diff changeset	58 return _urlopener.retrieve(url)
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	59 def urlcleanup():
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	60 if _urlopener:
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	61 _urlopener.cleanup()
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	62
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	63
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	64 # Class to open URLs.
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	65 # This is a class rather than just a subroutine because we may need
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	66 # more than one set of global protocol-specific options.
2899 65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	67 # Note -- this is a base class for those who don't want the
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	68 # automatic handling of errors type 302 (relocated) and 401
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	69 # (authorization needed).
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	70 ftpcache = {}
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	71 class URLopener:
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	72
4566 6403c94de50d Added safeguard against failure in __del__. Guido van Rossum <guido@python.org> parents: 4558 diff changeset	73 tempcache = None # So close() in __del__() won't fail
6403c94de50d Added safeguard against failure in __del__. Guido van Rossum <guido@python.org> parents: 4558 diff changeset	74
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	75 # Constructor
3461 803fbcdcfc9f Added proxy handling; upped version. Guido van Rossum <guido@python.org> parents: 3397 diff changeset	76 def __init__(self, proxies=None):
803fbcdcfc9f Added proxy handling; upped version. Guido van Rossum <guido@python.org> parents: 3397 diff changeset	77 if proxies is None:
803fbcdcfc9f Added proxy handling; upped version. Guido van Rossum <guido@python.org> parents: 3397 diff changeset	78 proxies = getproxies()
803fbcdcfc9f Added proxy handling; upped version. Guido van Rossum <guido@python.org> parents: 3397 diff changeset	79 self.proxies = proxies
2779 af7d0fea3fb2 add User-agent hdr; read and close the file upon http error Guido van Rossum <guido@python.org> parents: 2047 diff changeset	80 server_version = "Python-urllib/%s" % __version__
af7d0fea3fb2 add User-agent hdr; read and close the file upon http error Guido van Rossum <guido@python.org> parents: 2047 diff changeset	81 self.addheaders = [('User-agent', server_version)]
1810 d892ac4b4fb9 * Lib/linecache.py: don't crash on empty filename Guido van Rossum <guido@python.org> parents: 1778 diff changeset	82 self.tempcache = None
d892ac4b4fb9 * Lib/linecache.py: don't crash on empty filename Guido van Rossum <guido@python.org> parents: 1778 diff changeset	83 # Undocumented feature: if you assign {} to tempcache,
d892ac4b4fb9 * Lib/linecache.py: don't crash on empty filename Guido van Rossum <guido@python.org> parents: 1778 diff changeset	84 # it is used to cache files retrieved with
d892ac4b4fb9 * Lib/linecache.py: don't crash on empty filename Guido van Rossum <guido@python.org> parents: 1778 diff changeset	85 # self.retrieve(). This is not enabled by default
d892ac4b4fb9 * Lib/linecache.py: don't crash on empty filename Guido van Rossum <guido@python.org> parents: 1778 diff changeset	86 # since it does not work for changing documents (and I
d892ac4b4fb9 * Lib/linecache.py: don't crash on empty filename Guido van Rossum <guido@python.org> parents: 1778 diff changeset	87 # haven't got the logic to check expiration headers
d892ac4b4fb9 * Lib/linecache.py: don't crash on empty filename Guido van Rossum <guido@python.org> parents: 1778 diff changeset	88 # yet).
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	89 self.ftpcache = ftpcache
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	90 # Undocumented feature: you can use a different
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	91 # ftp cache by assigning to the .ftpcache member;
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	92 # in case you want logically independent URL openers
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	93
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	94 def __del__(self):
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	95 self.close()
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	96
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	97 def close(self):
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	98 self.cleanup()
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	99
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	100 def cleanup(self):
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	101 import os
1810 d892ac4b4fb9 * Lib/linecache.py: don't crash on empty filename Guido van Rossum <guido@python.org> parents: 1778 diff changeset	102 if self.tempcache:
d892ac4b4fb9 * Lib/linecache.py: don't crash on empty filename Guido van Rossum <guido@python.org> parents: 1778 diff changeset	103 for url in self.tempcache.keys():
d892ac4b4fb9 * Lib/linecache.py: don't crash on empty filename Guido van Rossum <guido@python.org> parents: 1778 diff changeset	104 try:
d892ac4b4fb9 * Lib/linecache.py: don't crash on empty filename Guido van Rossum <guido@python.org> parents: 1778 diff changeset	105 os.unlink(self.tempcache[url][0])
d892ac4b4fb9 * Lib/linecache.py: don't crash on empty filename Guido van Rossum <guido@python.org> parents: 1778 diff changeset	106 except os.error:
d892ac4b4fb9 * Lib/linecache.py: don't crash on empty filename Guido van Rossum <guido@python.org> parents: 1778 diff changeset	107 pass
d892ac4b4fb9 * Lib/linecache.py: don't crash on empty filename Guido van Rossum <guido@python.org> parents: 1778 diff changeset	108 del self.tempcache[url]
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	109
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	110 # Add a header to be used by the HTTP interface only
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	111 # e.g. u.addheader('Accept', 'sound/basic')
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	112 def addheader(self, *args):
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	113 self.addheaders.append(args)
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	114
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	115 # External interface
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	116 # Use URLopener().open(file) instead of open(file, 'r')
3028 7cd1da4064d7 support overriding how to open unknown url types Guido van Rossum <guido@python.org> parents: 2996 diff changeset	117 def open(self, fullurl):
7cd1da4064d7 support overriding how to open unknown url types Guido van Rossum <guido@python.org> parents: 2996 diff changeset	118 fullurl = unwrap(fullurl)
7cd1da4064d7 support overriding how to open unknown url types Guido van Rossum <guido@python.org> parents: 2996 diff changeset	119 type, url = splittype(fullurl)
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	120 if not type: type = 'file'
4257 239b10081176 One fix by sjoerd and one suggested by him. Bumped __version__ to 1.5. Guido van Rossum <guido@python.org> parents: 4153 diff changeset	121 self.openedurl = '%s:%s' % (type, url)
3461 803fbcdcfc9f Added proxy handling; upped version. Guido van Rossum <guido@python.org> parents: 3397 diff changeset	122 if self.proxies.has_key(type):
803fbcdcfc9f Added proxy handling; upped version. Guido van Rossum <guido@python.org> parents: 3397 diff changeset	123 proxy = self.proxies[type]
803fbcdcfc9f Added proxy handling; upped version. Guido van Rossum <guido@python.org> parents: 3397 diff changeset	124 type, proxy = splittype(proxy)
803fbcdcfc9f Added proxy handling; upped version. Guido van Rossum <guido@python.org> parents: 3397 diff changeset	125 host, selector = splithost(proxy)
803fbcdcfc9f Added proxy handling; upped version. Guido van Rossum <guido@python.org> parents: 3397 diff changeset	126 url = (host, fullurl) # Signal special case to open_*()
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	127 name = 'open_' + type
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	128 if '-' in name:
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	129 import regsub
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	130 name = regsub.gsub('-', '_', name)
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	131 if not hasattr(self, name):
3028 7cd1da4064d7 support overriding how to open unknown url types Guido van Rossum <guido@python.org> parents: 2996 diff changeset	132 return self.open_unknown(fullurl)
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	133 try:
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	134 return getattr(self, name)(url)
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	135 except socket.error, msg:
4558 df9d1ebcf7f3 When re-raising an exception raised by a module used internally as Guido van Rossum <guido@python.org> parents: 4532 diff changeset	136 raise IOError, ('socket error', msg), sys.exc_traceback
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	137
3028 7cd1da4064d7 support overriding how to open unknown url types Guido van Rossum <guido@python.org> parents: 2996 diff changeset	138 # Overridable interface to open unknown URL type
7cd1da4064d7 support overriding how to open unknown url types Guido van Rossum <guido@python.org> parents: 2996 diff changeset	139 def open_unknown(self, fullurl):
7cd1da4064d7 support overriding how to open unknown url types Guido van Rossum <guido@python.org> parents: 2996 diff changeset	140 type, url = splittype(fullurl)
7cd1da4064d7 support overriding how to open unknown url types Guido van Rossum <guido@python.org> parents: 2996 diff changeset	141 raise IOError, ('url error', 'unknown url type', type)
7cd1da4064d7 support overriding how to open unknown url types Guido van Rossum <guido@python.org> parents: 2996 diff changeset	142
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	143 # External interface
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	144 # retrieve(url) returns (filename, None) for a local object
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	145 # or (tempfilename, headers) for a remote object
3660 058d81d1d56d Support optional filename argument for retrieve() and urlretrieve(), Guido van Rossum <guido@python.org> parents: 3471 diff changeset	146 def retrieve(self, url, filename=None):
1810 d892ac4b4fb9 * Lib/linecache.py: don't crash on empty filename Guido van Rossum <guido@python.org> parents: 1778 diff changeset	147 if self.tempcache and self.tempcache.has_key(url):
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	148 return self.tempcache[url]
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	149 url1 = unwrap(url)
4440 fce296dbb03e Fix another case where... Guido van Rossum <guido@python.org> parents: 4395 diff changeset	150 self.openedurl = url1
1810 d892ac4b4fb9 * Lib/linecache.py: don't crash on empty filename Guido van Rossum <guido@python.org> parents: 1778 diff changeset	151 if self.tempcache and self.tempcache.has_key(url1):
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	152 self.tempcache[url] = self.tempcache[url1]
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	153 return self.tempcache[url1]
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	154 type, url1 = splittype(url1)
3660 058d81d1d56d Support optional filename argument for retrieve() and urlretrieve(), Guido van Rossum <guido@python.org> parents: 3471 diff changeset	155 if not filename and (not type or type == 'file'):
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	156 try:
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	157 fp = self.open_local_file(url1)
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	158 del fp
3303 d4d5c9fa7859 Added pathname2url and url2pathname methods (only correct for unix and Jack Jansen <jack.jansen@cwi.nl> parents: 3298 diff changeset	159 return url2pathname(splithost(url1)[1]), None
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	160 except IOError, msg:
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	161 pass
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	162 fp = self.open(url)
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	163 headers = fp.info()
3660 058d81d1d56d Support optional filename argument for retrieve() and urlretrieve(), Guido van Rossum <guido@python.org> parents: 3471 diff changeset	164 if not filename:
058d81d1d56d Support optional filename argument for retrieve() and urlretrieve(), Guido van Rossum <guido@python.org> parents: 3471 diff changeset	165 import tempfile
058d81d1d56d Support optional filename argument for retrieve() and urlretrieve(), Guido van Rossum <guido@python.org> parents: 3471 diff changeset	166 filename = tempfile.mktemp()
058d81d1d56d Support optional filename argument for retrieve() and urlretrieve(), Guido van Rossum <guido@python.org> parents: 3471 diff changeset	167 result = filename, headers
1810 d892ac4b4fb9 * Lib/linecache.py: don't crash on empty filename Guido van Rossum <guido@python.org> parents: 1778 diff changeset	168 if self.tempcache is not None:
1899 a6776a788f74 Fix bug if tmpcache is None Guido van Rossum <guido@python.org> parents: 1828 diff changeset	169 self.tempcache[url] = result
3660 058d81d1d56d Support optional filename argument for retrieve() and urlretrieve(), Guido van Rossum <guido@python.org> parents: 3471 diff changeset	170 tfp = open(filename, 'w')
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	171 bs = 1024*8
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	172 block = fp.read(bs)
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	173 while block:
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	174 tfp.write(block)
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	175 block = fp.read(bs)
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	176 del fp
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	177 del tfp
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	178 return result
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	179
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	180 # Each method named open_<type> knows how to open that type of URL
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	181
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	182 # Use HTTP protocol
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	183 def open_http(self, url):
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	184 import httplib
3461 803fbcdcfc9f Added proxy handling; upped version. Guido van Rossum <guido@python.org> parents: 3397 diff changeset	185 if type(url) is type(""):
803fbcdcfc9f Added proxy handling; upped version. Guido van Rossum <guido@python.org> parents: 3397 diff changeset	186 host, selector = splithost(url)
4153 cc06e70127fc Bump exposed __version__ to 1.4. Guido van Rossum <guido@python.org> parents: 4146 diff changeset	187 user_passwd, host = splituser(host)
3461 803fbcdcfc9f Added proxy handling; upped version. Guido van Rossum <guido@python.org> parents: 3397 diff changeset	188 else:
803fbcdcfc9f Added proxy handling; upped version. Guido van Rossum <guido@python.org> parents: 3397 diff changeset	189 host, selector = url
4153 cc06e70127fc Bump exposed __version__ to 1.4. Guido van Rossum <guido@python.org> parents: 4146 diff changeset	190 urltype, rest = splittype(selector)
cc06e70127fc Bump exposed __version__ to 1.4. Guido van Rossum <guido@python.org> parents: 4146 diff changeset	191 if string.lower(urltype) == 'http':
cc06e70127fc Bump exposed __version__ to 1.4. Guido van Rossum <guido@python.org> parents: 4146 diff changeset	192 realhost, rest = splithost(rest)
cc06e70127fc Bump exposed __version__ to 1.4. Guido van Rossum <guido@python.org> parents: 4146 diff changeset	193 user_passwd, realhost = splituser(realhost)
cc06e70127fc Bump exposed __version__ to 1.4. Guido van Rossum <guido@python.org> parents: 4146 diff changeset	194 if user_passwd:
cc06e70127fc Bump exposed __version__ to 1.4. Guido van Rossum <guido@python.org> parents: 4146 diff changeset	195 selector = "%s://%s%s" % (urltype,
cc06e70127fc Bump exposed __version__ to 1.4. Guido van Rossum <guido@python.org> parents: 4146 diff changeset	196 realhost, rest)
3461 803fbcdcfc9f Added proxy handling; upped version. Guido van Rossum <guido@python.org> parents: 3397 diff changeset	197 print "proxy via http:", host, selector
1590 ac2e5cc29949 Added tests for missing host to open_http and open_gopher Guido van Rossum <guido@python.org> parents: 1578 diff changeset	198 if not host: raise IOError, ('http error', 'no host given')
2899 65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	199 if user_passwd:
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	200 import base64
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	201 auth = string.strip(base64.encodestring(user_passwd))
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	202 else:
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	203 auth = None
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	204 h = httplib.HTTP(host)
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	205 h.putrequest('GET', selector)
4532 30ceca3f105b Fix the way the Authorization header is sent (how could this have worked?). Guido van Rossum <guido@python.org> parents: 4440 diff changeset	206 if auth: h.putheader('Authorization', 'Basic %s' % auth)
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	207 for args in self.addheaders: apply(h.putheader, args)
2779 af7d0fea3fb2 add User-agent hdr; read and close the file upon http error Guido van Rossum <guido@python.org> parents: 2047 diff changeset	208 h.endheaders()
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	209 errcode, errmsg, headers = h.getreply()
2779 af7d0fea3fb2 add User-agent hdr; read and close the file upon http error Guido van Rossum <guido@python.org> parents: 2047 diff changeset	210 fp = h.getfile()
af7d0fea3fb2 add User-agent hdr; read and close the file upon http error Guido van Rossum <guido@python.org> parents: 2047 diff changeset	211 if errcode == 200:
4257 239b10081176 One fix by sjoerd and one suggested by him. Bumped __version__ to 1.5. Guido van Rossum <guido@python.org> parents: 4153 diff changeset	212 return addinfourl(fp, headers, self.openedurl)
2779 af7d0fea3fb2 add User-agent hdr; read and close the file upon http error Guido van Rossum <guido@python.org> parents: 2047 diff changeset	213 else:
2899 65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	214 return self.http_error(url,
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	215 fp, errcode, errmsg, headers)
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	216
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	217 # Handle http errors.
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	218 # Derived class can override this, or provide specific handlers
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	219 # named http_error_DDD where DDD is the 3-digit error code
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	220 def http_error(self, url, fp, errcode, errmsg, headers):
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	221 # First check if there's a specific handler for this error
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	222 name = 'http_error_%d' % errcode
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	223 if hasattr(self, name):
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	224 method = getattr(self, name)
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	225 result = method(url, fp, errcode, errmsg, headers)
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	226 if result: return result
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	227 return self.http_error_default(
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	228 url, fp, errcode, errmsg, headers)
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	229
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	230 # Default http error handler: close the connection and raises IOError
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	231 def http_error_default(self, url, fp, errcode, errmsg, headers):
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	232 void = fp.read()
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	233 fp.close()
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	234 raise IOError, ('http error', errcode, errmsg, headers)
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	235
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	236 # Use Gopher protocol
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	237 def open_gopher(self, url):
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	238 import gopherlib
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	239 host, selector = splithost(url)
1590 ac2e5cc29949 Added tests for missing host to open_http and open_gopher Guido van Rossum <guido@python.org> parents: 1578 diff changeset	240 if not host: raise IOError, ('gopher error', 'no host given')
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	241 type, selector = splitgophertype(selector)
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	242 selector, query = splitquery(selector)
1701 f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	243 selector = unquote(selector)
f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	244 if query:
f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	245 query = unquote(query)
f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	246 fp = gopherlib.send_query(selector, query, host)
f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	247 else:
f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	248 fp = gopherlib.send_selector(selector, host)
4257 239b10081176 One fix by sjoerd and one suggested by him. Bumped __version__ to 1.5. Guido van Rossum <guido@python.org> parents: 4153 diff changeset	249 return addinfourl(fp, noheaders(), self.openedurl)
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	250
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	251 # Use local file or FTP depending on form of URL
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	252 def open_file(self, url):
3028 7cd1da4064d7 support overriding how to open unknown url types Guido van Rossum <guido@python.org> parents: 2996 diff changeset	253 if url[:2] == '//':
7cd1da4064d7 support overriding how to open unknown url types Guido van Rossum <guido@python.org> parents: 2996 diff changeset	254 return self.open_ftp(url)
7cd1da4064d7 support overriding how to open unknown url types Guido van Rossum <guido@python.org> parents: 2996 diff changeset	255 else:
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	256 return self.open_local_file(url)
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	257
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	258 # Use local file
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	259 def open_local_file(self, url):
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	260 host, file = splithost(url)
4395 48d1f5beb16b Fix some cases where self.openedurl wasn't set. Guido van Rossum <guido@python.org> parents: 4257 diff changeset	261 if not host:
48d1f5beb16b Fix some cases where self.openedurl wasn't set. Guido van Rossum <guido@python.org> parents: 4257 diff changeset	262 return addinfourl(open(url2pathname(file), 'r'), noheaders(), 'file:'+file)
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	263 host, port = splitport(host)
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	264 if not port and socket.gethostbyname(host) in (
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	265 localhost(), thishost()):
1701 f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	266 file = unquote(file)
4395 48d1f5beb16b Fix some cases where self.openedurl wasn't set. Guido van Rossum <guido@python.org> parents: 4257 diff changeset	267 return addinfourl(open(url2pathname(file), 'r'), noheaders(), 'file:'+file)
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	268 raise IOError, ('local file error', 'not on local host')
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	269
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	270 # Use FTP protocol
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	271 def open_ftp(self, url):
1701 f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	272 host, path = splithost(url)
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	273 if not host: raise IOError, ('ftp error', 'no host given')
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	274 host, port = splitport(host)
1701 f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	275 user, host = splituser(host)
f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	276 if user: user, passwd = splitpasswd(user)
f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	277 else: passwd = None
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	278 host = socket.gethostbyname(host)
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	279 if not port:
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	280 import ftplib
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	281 port = ftplib.FTP_PORT
1701 f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	282 path, attrs = splitattr(path)
f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	283 dirs = string.splitfields(path, '/')
f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	284 dirs, file = dirs[:-1], dirs[-1]
f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	285 if dirs and not dirs[0]: dirs = dirs[1:]
f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	286 key = (user, host, port, string.joinfields(dirs, '/'))
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	287 try:
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	288 if not self.ftpcache.has_key(key):
1701 f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	289 self.ftpcache[key] = \
f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	290 ftpwrapper(user, passwd,
f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	291 host, port, dirs)
f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	292 if not file: type = 'D'
f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	293 else: type = 'I'
f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	294 for attr in attrs:
f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	295 attr, value = splitvalue(attr)
f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	296 if string.lower(attr) == 'type' and \
f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	297 value in ('a', 'A', 'i', 'I', 'd', 'D'):
f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	298 type = string.upper(value)
4257 239b10081176 One fix by sjoerd and one suggested by him. Bumped __version__ to 1.5. Guido van Rossum <guido@python.org> parents: 4153 diff changeset	299 return addinfourl(self.ftpcache[key].retrfile(file, type),
239b10081176 One fix by sjoerd and one suggested by him. Bumped __version__ to 1.5. Guido van Rossum <guido@python.org> parents: 4153 diff changeset	300 noheaders(), self.openedurl)
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	301 except ftperrors(), msg:
4558 df9d1ebcf7f3 When re-raising an exception raised by a module used internally as Guido van Rossum <guido@python.org> parents: 4532 diff changeset	302 raise IOError, ('ftp error', msg), sys.exc_traceback
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	303
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	304
2899 65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	305 # Derived class with handlers for errors we can handle (perhaps)
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	306 class FancyURLopener(URLopener):
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	307
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	308 def __init__(self, *args):
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	309 apply(URLopener.__init__, (self,) + args)
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	310 self.auth_cache = {}
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	311
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	312 # Default error handling -- don't raise an exception
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	313 def http_error_default(self, url, fp, errcode, errmsg, headers):
4257 239b10081176 One fix by sjoerd and one suggested by him. Bumped __version__ to 1.5. Guido van Rossum <guido@python.org> parents: 4153 diff changeset	314 return addinfourl(fp, headers, self.openedurl)
2899 65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	315
4257 239b10081176 One fix by sjoerd and one suggested by him. Bumped __version__ to 1.5. Guido van Rossum <guido@python.org> parents: 4153 diff changeset	316 # Error 302 -- relocated (temporarily)
2899 65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	317 def http_error_302(self, url, fp, errcode, errmsg, headers):
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	318 # XXX The server can force infinite recursion here!
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	319 if headers.has_key('location'):
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	320 newurl = headers['location']
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	321 elif headers.has_key('uri'):
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	322 newurl = headers['uri']
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	323 else:
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	324 return
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	325 void = fp.read()
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	326 fp.close()
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	327 return self.open(newurl)
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	328
4257 239b10081176 One fix by sjoerd and one suggested by him. Bumped __version__ to 1.5. Guido van Rossum <guido@python.org> parents: 4153 diff changeset	329 # Error 301 -- also relocated (permanently)
239b10081176 One fix by sjoerd and one suggested by him. Bumped __version__ to 1.5. Guido van Rossum <guido@python.org> parents: 4153 diff changeset	330 http_error_301 = http_error_302
239b10081176 One fix by sjoerd and one suggested by him. Bumped __version__ to 1.5. Guido van Rossum <guido@python.org> parents: 4153 diff changeset	331
2899 65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	332 # Error 401 -- authentication required
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	333 # See this URL for a description of the basic authentication scheme:
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	334 # http://www.ics.uci.edu/pub/ietf/http/draft-ietf-http-v10-spec-00.txt
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	335 def http_error_401(self, url, fp, errcode, errmsg, headers):
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	336 if headers.has_key('www-authenticate'):
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	337 stuff = headers['www-authenticate']
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	338 p = regex.compile(
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	339 '[ \t]$[^ \t]+$[ \t]+realm="$[^"]$"')
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	340 if p.match(stuff) >= 0:
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	341 scheme, realm = p.group(1, 2)
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	342 if string.lower(scheme) == 'basic':
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	343 return self.retry_http_basic_auth(
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	344 url, realm)
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	345
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	346 def retry_http_basic_auth(self, url, realm):
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	347 host, selector = splithost(url)
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	348 i = string.find(host, '@') + 1
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	349 host = host[i:]
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	350 user, passwd = self.get_user_passwd(host, realm, i)
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	351 if not (user or passwd): return None
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	352 host = user + ':' + passwd + '@' + host
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	353 newurl = '//' + host + selector
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	354 return self.open_http(newurl)
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	355
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	356 def get_user_passwd(self, host, realm, clear_cache = 0):
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	357 key = realm + '@' + string.lower(host)
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	358 if self.auth_cache.has_key(key):
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	359 if clear_cache:
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	360 del self.auth_cache[key]
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	361 else:
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	362 return self.auth_cache[key]
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	363 user, passwd = self.prompt_user_passwd(host, realm)
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	364 if user or passwd: self.auth_cache[key] = (user, passwd)
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	365 return user, passwd
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	366
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	367 def prompt_user_passwd(self, host, realm):
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	368 # Override this in a GUI environment!
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	369 try:
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	370 user = raw_input("Enter username for %s at %s: " %
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	371 (realm, host))
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	372 self.echo_off()
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	373 try:
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	374 passwd = raw_input(
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	375 "Enter password for %s in %s at %s: " %
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	376 (user, realm, host))
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	377 finally:
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	378 self.echo_on()
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	379 return user, passwd
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	380 except KeyboardInterrupt:
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	381 return None, None
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	382
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	383 def echo_off(self):
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	384 import os
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	385 os.system("stty -echo")
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	386
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	387 def echo_on(self):
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	388 import os
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	389 print
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	390 os.system("stty echo")
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	391
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	392
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	393 # Utility functions
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	394
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	395 # Return the IP address of the magic hostname 'localhost'
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	396 _localhost = None
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	397 def localhost():
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	398 global _localhost
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	399 if not _localhost:
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	400 _localhost = socket.gethostbyname('localhost')
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	401 return _localhost
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	402
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	403 # Return the IP address of the current host
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	404 _thishost = None
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	405 def thishost():
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	406 global _thishost
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	407 if not _thishost:
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	408 _thishost = socket.gethostbyname(socket.gethostname())
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	409 return _thishost
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	410
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	411 # Return the set of errors raised by the FTP class
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	412 _ftperrors = None
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	413 def ftperrors():
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	414 global _ftperrors
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	415 if not _ftperrors:
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	416 import ftplib
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	417 _ftperrors = (ftplib.error_reply,
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	418 ftplib.error_temp,
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	419 ftplib.error_perm,
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	420 ftplib.error_proto)
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	421 return _ftperrors
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	422
2899 65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	423 # Return an empty mimetools.Message object
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	424 _noheaders = None
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	425 def noheaders():
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	426 global _noheaders
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	427 if not _noheaders:
2899 65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	428 import mimetools
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	429 import StringIO
65f06fc39960 use mimetools; add error handling and authentication Guido van Rossum <guido@python.org> parents: 2779 diff changeset	430 _noheaders = mimetools.Message(StringIO.StringIO(), 0)
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	431 _noheaders.fp.close() # Recycle file descriptor
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	432 return _noheaders
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	433
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	434
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	435 # Utility classes
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	436
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	437 # Class used by open_ftp() for cache of open FTP connections
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	438 class ftpwrapper:
1701 f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	439 def __init__(self, user, passwd, host, port, dirs):
f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	440 self.user = unquote(user or '')
f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	441 self.passwd = unquote(passwd or '')
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	442 self.host = host
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	443 self.port = port
1701 f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	444 self.dirs = []
f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	445 for dir in dirs:
f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	446 self.dirs.append(unquote(dir))
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	447 self.init()
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	448 def init(self):
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	449 import ftplib
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	450 self.ftp = ftplib.FTP()
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	451 self.ftp.connect(self.host, self.port)
1701 f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	452 self.ftp.login(self.user, self.passwd)
f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	453 for dir in self.dirs:
f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	454 self.ftp.cwd(dir)
f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	455 def retrfile(self, file, type):
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	456 import ftplib
1701 f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	457 if type in ('d', 'D'): cmd = 'TYPE A'; isdir = 1
f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	458 else: cmd = 'TYPE ' + type; isdir = 0
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	459 try:
1701 f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	460 self.ftp.voidcmd(cmd)
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	461 except ftplib.all_errors:
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	462 self.init()
1701 f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	463 self.ftp.voidcmd(cmd)
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	464 conn = None
1701 f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	465 if file and not isdir:
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	466 try:
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	467 cmd = 'RETR ' + file
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	468 conn = self.ftp.transfercmd(cmd)
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	469 except ftplib.error_perm, reason:
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	470 if reason[:3] != '550':
4558 df9d1ebcf7f3 When re-raising an exception raised by a module used internally as Guido van Rossum <guido@python.org> parents: 4532 diff changeset	471 raise IOError, ('ftp error', reason), \
df9d1ebcf7f3 When re-raising an exception raised by a module used internally as Guido van Rossum <guido@python.org> parents: 4532 diff changeset	472 sys.exc_traceback
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	473 if not conn:
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	474 # Try a directory listing
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	475 if file: cmd = 'LIST ' + file
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	476 else: cmd = 'LIST'
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	477 conn = self.ftp.transfercmd(cmd)
3397 5d7a15df7155 Try to normalize urls referring to local files (code copied from posixpath) Jack Jansen <jack.jansen@cwi.nl> parents: 3361 diff changeset	478 return addclosehook(conn.makefile('rb'), self.ftp.voidresp)
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	479
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	480 # Base class for addinfo and addclosehook
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	481 class addbase:
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	482 def __init__(self, fp):
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	483 self.fp = fp
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	484 self.read = self.fp.read
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	485 self.readline = self.fp.readline
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	486 self.readlines = self.fp.readlines
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	487 self.fileno = self.fp.fileno
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	488 def __repr__(self):
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	489 return '<%s at %s whose fp = %s>' % (
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	490 self.__class__.__name__, `id(self)`, `self.fp`)
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	491 def close(self):
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	492 self.read = None
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	493 self.readline = None
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	494 self.readlines = None
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	495 self.fileno = None
2779 af7d0fea3fb2 add User-agent hdr; read and close the file upon http error Guido van Rossum <guido@python.org> parents: 2047 diff changeset	496 if self.fp: self.fp.close()
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	497 self.fp = None
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	498
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	499 # Class to add a close hook to an open file
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	500 class addclosehook(addbase):
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	501 def __init__(self, fp, closehook, *hookargs):
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	502 addbase.__init__(self, fp)
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	503 self.closehook = closehook
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	504 self.hookargs = hookargs
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	505 def close(self):
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	506 if self.closehook:
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	507 apply(self.closehook, self.hookargs)
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	508 self.closehook = None
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	509 self.hookargs = None
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	510 addbase.close(self)
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	511
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	512 # class to add an info() method to an open file
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	513 class addinfo(addbase):
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	514 def __init__(self, fp, headers):
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	515 addbase.__init__(self, fp)
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	516 self.headers = headers
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	517 def info(self):
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	518 return self.headers
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	519
4257 239b10081176 One fix by sjoerd and one suggested by him. Bumped __version__ to 1.5. Guido van Rossum <guido@python.org> parents: 4153 diff changeset	520 # class to add info() and geturl() methods to an open file
239b10081176 One fix by sjoerd and one suggested by him. Bumped __version__ to 1.5. Guido van Rossum <guido@python.org> parents: 4153 diff changeset	521 class addinfourl(addbase):
239b10081176 One fix by sjoerd and one suggested by him. Bumped __version__ to 1.5. Guido van Rossum <guido@python.org> parents: 4153 diff changeset	522 def __init__(self, fp, headers, url):
239b10081176 One fix by sjoerd and one suggested by him. Bumped __version__ to 1.5. Guido van Rossum <guido@python.org> parents: 4153 diff changeset	523 addbase.__init__(self, fp)
239b10081176 One fix by sjoerd and one suggested by him. Bumped __version__ to 1.5. Guido van Rossum <guido@python.org> parents: 4153 diff changeset	524 self.headers = headers
239b10081176 One fix by sjoerd and one suggested by him. Bumped __version__ to 1.5. Guido van Rossum <guido@python.org> parents: 4153 diff changeset	525 self.url = url
239b10081176 One fix by sjoerd and one suggested by him. Bumped __version__ to 1.5. Guido van Rossum <guido@python.org> parents: 4153 diff changeset	526 def info(self):
239b10081176 One fix by sjoerd and one suggested by him. Bumped __version__ to 1.5. Guido van Rossum <guido@python.org> parents: 4153 diff changeset	527 return self.headers
239b10081176 One fix by sjoerd and one suggested by him. Bumped __version__ to 1.5. Guido van Rossum <guido@python.org> parents: 4153 diff changeset	528 def geturl(self):
239b10081176 One fix by sjoerd and one suggested by him. Bumped __version__ to 1.5. Guido van Rossum <guido@python.org> parents: 4153 diff changeset	529 return self.url
239b10081176 One fix by sjoerd and one suggested by him. Bumped __version__ to 1.5. Guido van Rossum <guido@python.org> parents: 4153 diff changeset	530
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	531
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	532 # Utility to combine a URL with a base URL to form a new URL
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	533
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	534 def basejoin(base, url):
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	535 type, path = splittype(url)
3244 2842ffa52126 Fixed basejoin. There were two main problems: Sjoerd Mullender <sjoerd@acm.org> parents: 3028 diff changeset	536 if type:
2842ffa52126 Fixed basejoin. There were two main problems: Sjoerd Mullender <sjoerd@acm.org> parents: 3028 diff changeset	537 # if url is complete (i.e., it contains a type), return it
2842ffa52126 Fixed basejoin. There were two main problems: Sjoerd Mullender <sjoerd@acm.org> parents: 3028 diff changeset	538 return url
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	539 host, path = splithost(path)
3244 2842ffa52126 Fixed basejoin. There were two main problems: Sjoerd Mullender <sjoerd@acm.org> parents: 3028 diff changeset	540 type, basepath = splittype(base) # inherit type from base
2842ffa52126 Fixed basejoin. There were two main problems: Sjoerd Mullender <sjoerd@acm.org> parents: 3028 diff changeset	541 if host:
2842ffa52126 Fixed basejoin. There were two main problems: Sjoerd Mullender <sjoerd@acm.org> parents: 3028 diff changeset	542 # if url contains host, just inherit type
2842ffa52126 Fixed basejoin. There were two main problems: Sjoerd Mullender <sjoerd@acm.org> parents: 3028 diff changeset	543 if type: return type + '://' + host + path
2842ffa52126 Fixed basejoin. There were two main problems: Sjoerd Mullender <sjoerd@acm.org> parents: 3028 diff changeset	544 else:
2842ffa52126 Fixed basejoin. There were two main problems: Sjoerd Mullender <sjoerd@acm.org> parents: 3028 diff changeset	545 # no type inherited, so url must have started with //
2842ffa52126 Fixed basejoin. There were two main problems: Sjoerd Mullender <sjoerd@acm.org> parents: 3028 diff changeset	546 # just return it
2842ffa52126 Fixed basejoin. There were two main problems: Sjoerd Mullender <sjoerd@acm.org> parents: 3028 diff changeset	547 return url
2842ffa52126 Fixed basejoin. There were two main problems: Sjoerd Mullender <sjoerd@acm.org> parents: 3028 diff changeset	548 host, basepath = splithost(basepath) # inherit host
2842ffa52126 Fixed basejoin. There were two main problems: Sjoerd Mullender <sjoerd@acm.org> parents: 3028 diff changeset	549 basepath, basetag = splittag(basepath) # remove extraneuous cruft
2842ffa52126 Fixed basejoin. There were two main problems: Sjoerd Mullender <sjoerd@acm.org> parents: 3028 diff changeset	550 basepath, basequery = splitquery(basepath) # idem
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	551 if path[:1] != '/':
3244 2842ffa52126 Fixed basejoin. There were two main problems: Sjoerd Mullender <sjoerd@acm.org> parents: 3028 diff changeset	552 # non-absolute path name
2842ffa52126 Fixed basejoin. There were two main problems: Sjoerd Mullender <sjoerd@acm.org> parents: 3028 diff changeset	553 if path[:1] in ('#', '?'):
2842ffa52126 Fixed basejoin. There were two main problems: Sjoerd Mullender <sjoerd@acm.org> parents: 3028 diff changeset	554 # path is just a tag or query, attach to basepath
2842ffa52126 Fixed basejoin. There were two main problems: Sjoerd Mullender <sjoerd@acm.org> parents: 3028 diff changeset	555 i = len(basepath)
2842ffa52126 Fixed basejoin. There were two main problems: Sjoerd Mullender <sjoerd@acm.org> parents: 3028 diff changeset	556 else:
2842ffa52126 Fixed basejoin. There were two main problems: Sjoerd Mullender <sjoerd@acm.org> parents: 3028 diff changeset	557 # else replace last component
2842ffa52126 Fixed basejoin. There were two main problems: Sjoerd Mullender <sjoerd@acm.org> parents: 3028 diff changeset	558 i = string.rfind(basepath, '/')
2842ffa52126 Fixed basejoin. There were two main problems: Sjoerd Mullender <sjoerd@acm.org> parents: 3028 diff changeset	559 if i < 0:
2842ffa52126 Fixed basejoin. There were two main problems: Sjoerd Mullender <sjoerd@acm.org> parents: 3028 diff changeset	560 # basepath not absolute
2842ffa52126 Fixed basejoin. There were two main problems: Sjoerd Mullender <sjoerd@acm.org> parents: 3028 diff changeset	561 if host:
2842ffa52126 Fixed basejoin. There were two main problems: Sjoerd Mullender <sjoerd@acm.org> parents: 3028 diff changeset	562 # host present, make absolute
2842ffa52126 Fixed basejoin. There were two main problems: Sjoerd Mullender <sjoerd@acm.org> parents: 3028 diff changeset	563 basepath = '/'
2842ffa52126 Fixed basejoin. There were two main problems: Sjoerd Mullender <sjoerd@acm.org> parents: 3028 diff changeset	564 else:
2842ffa52126 Fixed basejoin. There were two main problems: Sjoerd Mullender <sjoerd@acm.org> parents: 3028 diff changeset	565 # else keep non-absolute
2842ffa52126 Fixed basejoin. There were two main problems: Sjoerd Mullender <sjoerd@acm.org> parents: 3028 diff changeset	566 basepath = ''
2842ffa52126 Fixed basejoin. There were two main problems: Sjoerd Mullender <sjoerd@acm.org> parents: 3028 diff changeset	567 else:
2842ffa52126 Fixed basejoin. There were two main problems: Sjoerd Mullender <sjoerd@acm.org> parents: 3028 diff changeset	568 # remove last file component
2842ffa52126 Fixed basejoin. There were two main problems: Sjoerd Mullender <sjoerd@acm.org> parents: 3028 diff changeset	569 basepath = basepath[:i+1]
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	570 path = basepath + path
3244 2842ffa52126 Fixed basejoin. There were two main problems: Sjoerd Mullender <sjoerd@acm.org> parents: 3028 diff changeset	571 if type and host: return type + '://' + host + path
2842ffa52126 Fixed basejoin. There were two main problems: Sjoerd Mullender <sjoerd@acm.org> parents: 3028 diff changeset	572 elif type: return type + ':' + path
2842ffa52126 Fixed basejoin. There were two main problems: Sjoerd Mullender <sjoerd@acm.org> parents: 3028 diff changeset	573 elif host: return '//' + host + path # don't know what this means
2842ffa52126 Fixed basejoin. There were two main problems: Sjoerd Mullender <sjoerd@acm.org> parents: 3028 diff changeset	574 else: return path
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	575
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	576
1701 f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	577 # Utilities to parse URLs (most of these return None for missing parts):
3244 2842ffa52126 Fixed basejoin. There were two main problems: Sjoerd Mullender <sjoerd@acm.org> parents: 3028 diff changeset	578 # unwrap('<URL:type://host/path>') --> 'type://host/path'
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	579 # splittype('type:opaquestring') --> 'type', 'opaquestring'
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	580 # splithost('//host[:port]/path') --> 'host[:port]', '/path'
1701 f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	581 # splituser('user[:passwd]@host[:port]') --> 'user[:passwd]', 'host[:port]'
f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	582 # splitpasswd('user:passwd') -> 'user', 'passwd'
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	583 # splitport('host:port') --> 'host', 'port'
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	584 # splitquery('/path?query') --> '/path', 'query'
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	585 # splittag('/path#tag') --> '/path', 'tag'
1701 f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	586 # splitattr('/path;attr1=value1;attr2=value2;...') ->
f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	587 # '/path', ['attr1=value1', 'attr2=value2', ...]
f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	588 # splitvalue('attr=value') --> 'attr', 'value'
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	589 # splitgophertype('/Xselector') --> 'X', 'selector'
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	590 # unquote('abc%20def') -> 'abc def'
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	591 # quote('abc def') -> 'abc%20def')
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	592
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	593 def unwrap(url):
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	594 url = string.strip(url)
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	595 if url[:1] == '<' and url[-1:] == '>':
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	596 url = string.strip(url[1:-1])
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	597 if url[:4] == 'URL:': url = string.strip(url[4:])
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	598 return url
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	599
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	600 _typeprog = regex.compile('^$[^/:]+$:$.*$$')
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	601 def splittype(url):
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	602 if _typeprog.match(url) >= 0: return _typeprog.group(1, 2)
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	603 return None, url
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	604
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	605 _hostprog = regex.compile('^//$[^/]+$$.*$$')
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	606 def splithost(url):
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	607 if _hostprog.match(url) >= 0: return _hostprog.group(1, 2)
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	608 return None, url
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	609
1701 f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	610 _userprog = regex.compile('^$[^@]$@$.$$')
f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	611 def splituser(host):
f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	612 if _userprog.match(host) >= 0: return _userprog.group(1, 2)
f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	613 return None, host
f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	614
f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	615 _passwdprog = regex.compile('^$[^:]$:$.$$')
f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	616 def splitpasswd(user):
f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	617 if _passwdprog.match(user) >= 0: return _passwdprog.group(1, 2)
f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	618 return user, None
f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	619
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	620 _portprog = regex.compile('^$.*$:$[0-9]+$$')
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	621 def splitport(host):
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	622 if _portprog.match(host) >= 0: return _portprog.group(1, 2)
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	623 return host, None
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	624
3677 b05315816579 Added splitnport(), which is like splitport() but returns a numeric port, Guido van Rossum <guido@python.org> parents: 3660 diff changeset	625 # Split host and port, returning numeric port.
b05315816579 Added splitnport(), which is like splitport() but returns a numeric port, Guido van Rossum <guido@python.org> parents: 3660 diff changeset	626 # Return given default port if no ':' found; defaults to -1.
3687 d8f9dfc76bbf Change defn of splitnport() to only accept valid digit strings. Guido van Rossum <guido@python.org> parents: 3677 diff changeset	627 # Return numerical port if a valid number are found after ':'.
d8f9dfc76bbf Change defn of splitnport() to only accept valid digit strings. Guido van Rossum <guido@python.org> parents: 3677 diff changeset	628 # Return None if ':' but not a valid number.
d8f9dfc76bbf Change defn of splitnport() to only accept valid digit strings. Guido van Rossum <guido@python.org> parents: 3677 diff changeset	629 _nportprog = regex.compile('^$.$:$.$$')
3677 b05315816579 Added splitnport(), which is like splitport() but returns a numeric port, Guido van Rossum <guido@python.org> parents: 3660 diff changeset	630 def splitnport(host, defport=-1):
b05315816579 Added splitnport(), which is like splitport() but returns a numeric port, Guido van Rossum <guido@python.org> parents: 3660 diff changeset	631 if _nportprog.match(host) >= 0:
3687 d8f9dfc76bbf Change defn of splitnport() to only accept valid digit strings. Guido van Rossum <guido@python.org> parents: 3677 diff changeset	632 host, port = _nportprog.group(1, 2)
d8f9dfc76bbf Change defn of splitnport() to only accept valid digit strings. Guido van Rossum <guido@python.org> parents: 3677 diff changeset	633 try:
d8f9dfc76bbf Change defn of splitnport() to only accept valid digit strings. Guido van Rossum <guido@python.org> parents: 3677 diff changeset	634 if not port: raise string.atoi_error, "no digits"
d8f9dfc76bbf Change defn of splitnport() to only accept valid digit strings. Guido van Rossum <guido@python.org> parents: 3677 diff changeset	635 nport = string.atoi(port)
d8f9dfc76bbf Change defn of splitnport() to only accept valid digit strings. Guido van Rossum <guido@python.org> parents: 3677 diff changeset	636 except string.atoi_error:
d8f9dfc76bbf Change defn of splitnport() to only accept valid digit strings. Guido van Rossum <guido@python.org> parents: 3677 diff changeset	637 nport = None
3677 b05315816579 Added splitnport(), which is like splitport() but returns a numeric port, Guido van Rossum <guido@python.org> parents: 3660 diff changeset	638 return host, nport
b05315816579 Added splitnport(), which is like splitport() but returns a numeric port, Guido van Rossum <guido@python.org> parents: 3660 diff changeset	639 return host, defport
b05315816579 Added splitnport(), which is like splitport() but returns a numeric port, Guido van Rossum <guido@python.org> parents: 3660 diff changeset	640
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	641 _queryprog = regex.compile('^$.$\?$[^?]$$')
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	642 def splitquery(url):
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	643 if _queryprog.match(url) >= 0: return _queryprog.group(1, 2)
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	644 return url, None
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	645
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	646 _tagprog = regex.compile('^$.$#$[^#]$$')
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	647 def splittag(url):
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	648 if _tagprog.match(url) >= 0: return _tagprog.group(1, 2)
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	649 return url, None
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	650
1701 f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	651 def splitattr(url):
f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	652 words = string.splitfields(url, ';')
f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	653 return words[0], words[1:]
f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	654
f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	655 _valueprog = regex.compile('^$[^=]$=$.$$')
f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	656 def splitvalue(attr):
f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	657 if _valueprog.match(attr) >= 0: return _valueprog.group(1, 2)
f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	658 return attr, None
f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	659
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	660 def splitgophertype(selector):
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	661 if selector[:1] == '/' and selector[1:2]:
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	662 return selector[1], selector[2:]
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	663 return None, selector
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	664
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	665 _quoteprog = regex.compile('%[0-9a-fA-F][0-9a-fA-F]')
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	666 def unquote(s):
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	667 i = 0
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	668 n = len(s)
4146 c4d66c410c83 Slightly faster (un)quoting. Guido van Rossum <guido@python.org> parents: 3724 diff changeset	669 res = []
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	670 while 0 <= i < n:
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	671 j = _quoteprog.search(s, i)
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	672 if j < 0:
4146 c4d66c410c83 Slightly faster (un)quoting. Guido van Rossum <guido@python.org> parents: 3724 diff changeset	673 res.append(s[i:])
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	674 break
4146 c4d66c410c83 Slightly faster (un)quoting. Guido van Rossum <guido@python.org> parents: 3724 diff changeset	675 res.append(s[i:j] + chr(string.atoi(s[j+1:j+3], 16)))
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	676 i = j+3
4146 c4d66c410c83 Slightly faster (un)quoting. Guido van Rossum <guido@python.org> parents: 3724 diff changeset	677 return string.joinfields(res, '')
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	678
1828 b464e1d0b2fb New way of generating .pyc files, thanks to Sjoerd. Guido van Rossum <guido@python.org> parents: 1810 diff changeset	679 always_safe = string.letters + string.digits + '_,.-'
1701 f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	680 def quote(s, safe = '/'):
f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	681 safe = always_safe + safe
4146 c4d66c410c83 Slightly faster (un)quoting. Guido van Rossum <guido@python.org> parents: 3724 diff changeset	682 res = []
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	683 for c in s:
1701 f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	684 if c in safe:
4146 c4d66c410c83 Slightly faster (un)quoting. Guido van Rossum <guido@python.org> parents: 3724 diff changeset	685 res.append(c)
1701 f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	686 else:
4146 c4d66c410c83 Slightly faster (un)quoting. Guido van Rossum <guido@python.org> parents: 3724 diff changeset	687 res.append('%%%02x' % ord(c))
c4d66c410c83 Slightly faster (un)quoting. Guido van Rossum <guido@python.org> parents: 3724 diff changeset	688 return string.joinfields(res, '')
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	689
3461 803fbcdcfc9f Added proxy handling; upped version. Guido van Rossum <guido@python.org> parents: 3397 diff changeset	690
803fbcdcfc9f Added proxy handling; upped version. Guido van Rossum <guido@python.org> parents: 3397 diff changeset	691 # Proxy handling
803fbcdcfc9f Added proxy handling; upped version. Guido van Rossum <guido@python.org> parents: 3397 diff changeset	692 def getproxies():
803fbcdcfc9f Added proxy handling; upped version. Guido van Rossum <guido@python.org> parents: 3397 diff changeset	693 """Return a dictionary of protocol scheme -> proxy server URL mappings.
803fbcdcfc9f Added proxy handling; upped version. Guido van Rossum <guido@python.org> parents: 3397 diff changeset	694
803fbcdcfc9f Added proxy handling; upped version. Guido van Rossum <guido@python.org> parents: 3397 diff changeset	695 Scan the environment for variables named <scheme>_proxy;
803fbcdcfc9f Added proxy handling; upped version. Guido van Rossum <guido@python.org> parents: 3397 diff changeset	696 this seems to be the standard convention. If you need a
803fbcdcfc9f Added proxy handling; upped version. Guido van Rossum <guido@python.org> parents: 3397 diff changeset	697 different way, you can pass a proxies dictionary to the
803fbcdcfc9f Added proxy handling; upped version. Guido van Rossum <guido@python.org> parents: 3397 diff changeset	698 [Fancy]URLopener constructor.
803fbcdcfc9f Added proxy handling; upped version. Guido van Rossum <guido@python.org> parents: 3397 diff changeset	699
803fbcdcfc9f Added proxy handling; upped version. Guido van Rossum <guido@python.org> parents: 3397 diff changeset	700 """
803fbcdcfc9f Added proxy handling; upped version. Guido van Rossum <guido@python.org> parents: 3397 diff changeset	701 proxies = {}
803fbcdcfc9f Added proxy handling; upped version. Guido van Rossum <guido@python.org> parents: 3397 diff changeset	702 for name, value in os.environ.items():
803fbcdcfc9f Added proxy handling; upped version. Guido van Rossum <guido@python.org> parents: 3397 diff changeset	703 if value and name[-6:] == '_proxy':
803fbcdcfc9f Added proxy handling; upped version. Guido van Rossum <guido@python.org> parents: 3397 diff changeset	704 proxies[name[:-6]] = value
803fbcdcfc9f Added proxy handling; upped version. Guido van Rossum <guido@python.org> parents: 3397 diff changeset	705 return proxies
803fbcdcfc9f Added proxy handling; upped version. Guido van Rossum <guido@python.org> parents: 3397 diff changeset	706
803fbcdcfc9f Added proxy handling; upped version. Guido van Rossum <guido@python.org> parents: 3397 diff changeset	707
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	708 # Test and time quote() and unquote()
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	709 def test1():
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	710 import time
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	711 s = ''
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	712 for i in range(256): s = s + chr(i)
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	713 s = s*4
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	714 t0 = time.time()
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	715 qs = quote(s)
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	716 uqs = unquote(qs)
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	717 t1 = time.time()
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	718 if uqs != s:
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	719 print 'Wrong!'
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	720 print `s`
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	721 print `qs`
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	722 print `uqs`
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	723 print round(t1 - t0, 3), 'sec'
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	724
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	725
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	726 # Test program
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	727 def test():
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	728 import sys
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	729 import regsub
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	730 args = sys.argv[1:]
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	731 if not args:
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	732 args = [
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	733 '/etc/passwd',
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	734 'file:/etc/passwd',
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	735 'file://localhost/etc/passwd',
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	736 'ftp://ftp.cwi.nl/etc/passwd',
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	737 'gopher://gopher.cwi.nl/11/',
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	738 'http://www.cwi.nl/index.html',
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	739 ]
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	740 try:
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	741 for url in args:
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	742 print '-'10, url, '-'10
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	743 fn, h = urlretrieve(url)
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	744 print fn, h
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	745 if h:
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	746 print '======'
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	747 for k in h.keys(): print k + ':', h[k]
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	748 print '======'
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	749 fp = open(fn, 'r')
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	750 data = fp.read()
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	751 del fp
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	752 print regsub.gsub('\r', '', data)
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	753 fn, h = None, None
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	754 print '-'*40
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	755 finally:
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	756 urlcleanup()
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	757
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	758 # Run test program when run as a script
b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	759 if __name__ == '__main__':
1701 f1b040bca3e1 * Lib/urllib.py: implemented new quoting rules; added splituser, Guido van Rossum <guido@python.org> parents: 1590 diff changeset	760 ## test1()
1578 b21f048c23e9 Renamed urlopen.py to urllib.py. Guido van Rossum <guido@python.org> parents: diff changeset	761 test()

Mercurial > cpython-lru_cache-weakref

annotate Lib/urllib.py @ 4566:6403c94de50d legacy-trunk