Mercurial > cpython-lru_cache-weakref

annotate Lib/httplib.py @ 58765:2ba7677bfa95 legacy-trunk

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
testCopysign was defined twice in test_math; combine the definitions
author Mark Dickinson <dickinsm@gmail.com>
date Sat, 06 Feb 2010 23:11:25 +0000
parents a19ad299095f
children 45212a247ed4
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1 """HTTP/1.1 client library
6787
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
2
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
3 <intro stuff goes here>
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
4 <other stuff, too>
6787
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
5
39293
efbaec9674df Minor typo fixes
Andrew M. Kuchling <amk@amk.ca>
parents: 39190
diff changeset
6 HTTPConnection goes through a number of "states", which define when a client
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
7 may legally make another request or fetch the response for a particular
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
8 request. This diagram details these state transitions:
1555
6d35308261f4 Towards a standard access mechanism for URLs.
Guido van Rossum <guido@python.org>
parents:
diff changeset
9
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
10 (null)
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
11 |
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
12 | HTTPConnection()
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
13 v
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
14 Idle
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
15 |
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
16 | putrequest()
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
17 v
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
18 Request-started
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
19 |
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
20 | ( putheader() )* endheaders()
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
21 v
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
22 Request-sent
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
23 |
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
24 | response = getresponse()
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
25 v
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
26 Unread-response [Response-headers-read]
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
27 |\____________________
15845
210bbe53b6d4 No text file relying on significant trailing whitespace is robust under
Tim Peters <tim.peters@gmail.com>
parents: 15844
diff changeset
28 | |
210bbe53b6d4 No text file relying on significant trailing whitespace is robust under
Tim Peters <tim.peters@gmail.com>
parents: 15844
diff changeset
29 | response.read() | putrequest()
210bbe53b6d4 No text file relying on significant trailing whitespace is robust under
Tim Peters <tim.peters@gmail.com>
parents: 15844
diff changeset
30 v v
210bbe53b6d4 No text file relying on significant trailing whitespace is robust under
Tim Peters <tim.peters@gmail.com>
parents: 15844
diff changeset
31 Idle Req-started-unread-response
210bbe53b6d4 No text file relying on significant trailing whitespace is robust under
Tim Peters <tim.peters@gmail.com>
parents: 15844
diff changeset
32 ______/|
210bbe53b6d4 No text file relying on significant trailing whitespace is robust under
Tim Peters <tim.peters@gmail.com>
parents: 15844
diff changeset
33 / |
210bbe53b6d4 No text file relying on significant trailing whitespace is robust under
Tim Peters <tim.peters@gmail.com>
parents: 15844
diff changeset
34 response.read() | | ( putheader() )* endheaders()
210bbe53b6d4 No text file relying on significant trailing whitespace is robust under
Tim Peters <tim.peters@gmail.com>
parents: 15844
diff changeset
35 v v
210bbe53b6d4 No text file relying on significant trailing whitespace is robust under
Tim Peters <tim.peters@gmail.com>
parents: 15844
diff changeset
36 Request-started Req-sent-unread-response
210bbe53b6d4 No text file relying on significant trailing whitespace is robust under
Tim Peters <tim.peters@gmail.com>
parents: 15844
diff changeset
37 |
210bbe53b6d4 No text file relying on significant trailing whitespace is robust under
Tim Peters <tim.peters@gmail.com>
parents: 15844
diff changeset
38 | response.read()
210bbe53b6d4 No text file relying on significant trailing whitespace is robust under
Tim Peters <tim.peters@gmail.com>
parents: 15844
diff changeset
39 v
210bbe53b6d4 No text file relying on significant trailing whitespace is robust under
Tim Peters <tim.peters@gmail.com>
parents: 15844
diff changeset
40 Request-sent
6787
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
41
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
42 This diagram presents the following rules:
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
43 -- a second request may not be started until {response-headers-read}
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
44 -- a response [object] cannot be retrieved until {request-sent}
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
45 -- there is no differentiation between an unread response body and a
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
46 partially read response body
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
47
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
48 Note: this enforcement is applied by the HTTPConnection class. The
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
49 HTTPResponse class does not enforce this state machine, which
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
50 implies sophisticated clients may accelerate the request/response
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
51 pipeline. Caution should be taken, though: accelerating the states
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
52 beyond the above pattern may imply knowledge of the server's
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
53 connection-close behavior for certain requests. For example, it
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
54 is impossible to tell whether the server will close the connection
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
55 UNTIL the response headers have been read; this means that further
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
56 requests cannot be placed into the pipeline until it is known that
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
57 the server will NOT be closing the connection.
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
58
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
59 Logical State __state __response
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
60 ------------- ------- ----------
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
61 Idle _CS_IDLE None
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
62 Request-started _CS_REQ_STARTED None
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
63 Request-sent _CS_REQ_SENT None
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
64 Unread-response _CS_IDLE <response_class>
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
65 Req-started-unread-response _CS_REQ_STARTED <response_class>
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
66 Req-sent-unread-response _CS_REQ_SENT <response_class>
6787
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
67 """
1555
6d35308261f4 Towards a standard access mechanism for URLs.
Guido van Rossum <guido@python.org>
parents:
diff changeset
68
56285
e0f234116ab1 Issue #6790: Make it possible again to pass an `array.array` to
Antoine Pitrou <solipsis@pitrou.net>
parents: 55827
diff changeset
69 from array import array
1555
6d35308261f4 Towards a standard access mechanism for URLs.
Guido van Rossum <guido@python.org>
parents:
diff changeset
70 import socket
48948
d563621d76db Move test.test_support.catch_warning() to the warnings module, rename it
Brett Cannon <bcannon@gmail.com>
parents: 48249
diff changeset
71 from sys import py3kwarning
22192
044f9e71320f SF bug report #405939: wrong Host header with proxy
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 21962
diff changeset
72 from urlparse import urlsplit
43161
9cfad9a5c83b deprecate use of FakeSocket
Bill Janssen <janssen@parc.com>
parents: 43093
diff changeset
73 import warnings
48948
d563621d76db Move test.test_support.catch_warning() to the warnings module, rename it
Brett Cannon <bcannon@gmail.com>
parents: 48249
diff changeset
74 with warnings.catch_warnings():
d563621d76db Move test.test_support.catch_warning() to the warnings module, rename it
Brett Cannon <bcannon@gmail.com>
parents: 48249
diff changeset
75 if py3kwarning:
d563621d76db Move test.test_support.catch_warning() to the warnings module, rename it
Brett Cannon <bcannon@gmail.com>
parents: 48249
diff changeset
76 warnings.filterwarnings("ignore", ".*mimetools has been removed",
d563621d76db Move test.test_support.catch_warning() to the warnings module, rename it
Brett Cannon <bcannon@gmail.com>
parents: 48249
diff changeset
77 DeprecationWarning)
48249
918a918a9996 Silence the DeprecationWarning raised in httplib when mimetools is imported.
Brett Cannon <bcannon@gmail.com>
parents: 47909
diff changeset
78 import mimetools
1555
6d35308261f4 Towards a standard access mechanism for URLs.
Guido van Rossum <guido@python.org>
parents:
diff changeset
79
11527
28b8585354de OpenSSL support. This is based on patches for a version of SSLeay by
Guido van Rossum <guido@python.org>
parents: 7122
diff changeset
80 try:
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
81 from cStringIO import StringIO
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
82 except ImportError:
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
83 from StringIO import StringIO
11527
28b8585354de OpenSSL support. This is based on patches for a version of SSLeay by
Guido van Rossum <guido@python.org>
parents: 7122
diff changeset
84
43093
1d57fb7efb15 Don't lie in __all__ attributes when SSL is not available: only add the SSL
Thomas Wouters <thomas@python.org>
parents: 43008
diff changeset
85 __all__ = ["HTTP", "HTTPResponse", "HTTPConnection",
18013
99aea6f5cebd more public symbols for __all__
Skip Montanaro <skip@pobox.com>
parents: 17482
diff changeset
86 "HTTPException", "NotConnected", "UnknownProtocol",
24042
63fbbed10714 Simplify HTTPSConnection constructor.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24038
diff changeset
87 "UnknownTransferEncoding", "UnimplementedFileMode",
63fbbed10714 Simplify HTTPSConnection constructor.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24038
diff changeset
88 "IncompleteRead", "InvalidURL", "ImproperConnectionState",
63fbbed10714 Simplify HTTPSConnection constructor.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24038
diff changeset
89 "CannotSendRequest", "CannotSendHeader", "ResponseNotReady",
36559
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
90 "BadStatusLine", "error", "responses"]
16160
46920b60cd33 added a few more __all__ lists
Skip Montanaro <skip@pobox.com>
parents: 15878
diff changeset
91
1555
6d35308261f4 Towards a standard access mechanism for URLs.
Guido van Rossum <guido@python.org>
parents:
diff changeset
92 HTTP_PORT = 80
11527
28b8585354de OpenSSL support. This is based on patches for a version of SSLeay by
Guido van Rossum <guido@python.org>
parents: 7122
diff changeset
93 HTTPS_PORT = 443
28b8585354de OpenSSL support. This is based on patches for a version of SSLeay by
Guido van Rossum <guido@python.org>
parents: 7122
diff changeset
94
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
95 _UNKNOWN = 'UNKNOWN'
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
96
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
97 # connection states
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
98 _CS_IDLE = 'Idle'
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
99 _CS_REQ_STARTED = 'Request-started'
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
100 _CS_REQ_SENT = 'Request-sent'
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
101
33506
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
102 # status codes
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
103 # informational
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
104 CONTINUE = 100
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
105 SWITCHING_PROTOCOLS = 101
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
106 PROCESSING = 102
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
107
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
108 # successful
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
109 OK = 200
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
110 CREATED = 201
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
111 ACCEPTED = 202
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
112 NON_AUTHORITATIVE_INFORMATION = 203
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
113 NO_CONTENT = 204
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
114 RESET_CONTENT = 205
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
115 PARTIAL_CONTENT = 206
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
116 MULTI_STATUS = 207
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
117 IM_USED = 226
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
118
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
119 # redirection
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
120 MULTIPLE_CHOICES = 300
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
121 MOVED_PERMANENTLY = 301
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
122 FOUND = 302
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
123 SEE_OTHER = 303
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
124 NOT_MODIFIED = 304
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
125 USE_PROXY = 305
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
126 TEMPORARY_REDIRECT = 307
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
127
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
128 # client error
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
129 BAD_REQUEST = 400
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
130 UNAUTHORIZED = 401
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
131 PAYMENT_REQUIRED = 402
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
132 FORBIDDEN = 403
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
133 NOT_FOUND = 404
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
134 METHOD_NOT_ALLOWED = 405
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
135 NOT_ACCEPTABLE = 406
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
136 PROXY_AUTHENTICATION_REQUIRED = 407
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
137 REQUEST_TIMEOUT = 408
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
138 CONFLICT = 409
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
139 GONE = 410
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
140 LENGTH_REQUIRED = 411
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
141 PRECONDITION_FAILED = 412
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
142 REQUEST_ENTITY_TOO_LARGE = 413
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
143 REQUEST_URI_TOO_LONG = 414
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
144 UNSUPPORTED_MEDIA_TYPE = 415
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
145 REQUESTED_RANGE_NOT_SATISFIABLE = 416
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
146 EXPECTATION_FAILED = 417
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
147 UNPROCESSABLE_ENTITY = 422
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
148 LOCKED = 423
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
149 FAILED_DEPENDENCY = 424
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
150 UPGRADE_REQUIRED = 426
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
151
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
152 # server error
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
153 INTERNAL_SERVER_ERROR = 500
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
154 NOT_IMPLEMENTED = 501
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
155 BAD_GATEWAY = 502
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
156 SERVICE_UNAVAILABLE = 503
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
157 GATEWAY_TIMEOUT = 504
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
158 HTTP_VERSION_NOT_SUPPORTED = 505
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
159 INSUFFICIENT_STORAGE = 507
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
160 NOT_EXTENDED = 510
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
161
36559
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
162 # Mapping status codes to official W3C names
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
163 responses = {
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
164 100: 'Continue',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
165 101: 'Switching Protocols',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
166
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
167 200: 'OK',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
168 201: 'Created',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
169 202: 'Accepted',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
170 203: 'Non-Authoritative Information',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
171 204: 'No Content',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
172 205: 'Reset Content',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
173 206: 'Partial Content',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
174
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
175 300: 'Multiple Choices',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
176 301: 'Moved Permanently',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
177 302: 'Found',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
178 303: 'See Other',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
179 304: 'Not Modified',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
180 305: 'Use Proxy',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
181 306: '(Unused)',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
182 307: 'Temporary Redirect',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
183
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
184 400: 'Bad Request',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
185 401: 'Unauthorized',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
186 402: 'Payment Required',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
187 403: 'Forbidden',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
188 404: 'Not Found',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
189 405: 'Method Not Allowed',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
190 406: 'Not Acceptable',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
191 407: 'Proxy Authentication Required',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
192 408: 'Request Timeout',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
193 409: 'Conflict',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
194 410: 'Gone',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
195 411: 'Length Required',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
196 412: 'Precondition Failed',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
197 413: 'Request Entity Too Large',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
198 414: 'Request-URI Too Long',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
199 415: 'Unsupported Media Type',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
200 416: 'Requested Range Not Satisfiable',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
201 417: 'Expectation Failed',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
202
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
203 500: 'Internal Server Error',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
204 501: 'Not Implemented',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
205 502: 'Bad Gateway',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
206 503: 'Service Unavailable',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
207 504: 'Gateway Timeout',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
208 505: 'HTTP Version Not Supported',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
209 }
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
210
35652
6a20831d357d bug [ 1296004 ] MemoryError in httplib
Georg Brandl <georg@python.org>
parents: 35158
diff changeset
211 # maximal amount of data to read at one time in _safe_read
6a20831d357d bug [ 1296004 ] MemoryError in httplib
Georg Brandl <georg@python.org>
parents: 35158
diff changeset
212 MAXAMOUNT = 1048576
6a20831d357d bug [ 1296004 ] MemoryError in httplib
Georg Brandl <georg@python.org>
parents: 35158
diff changeset
213
24123
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
214 class HTTPMessage(mimetools.Message):
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
215
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
216 def addheader(self, key, value):
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
217 """Add header for field key handling repeats."""
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
218 prev = self.dict.get(key)
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
219 if prev is None:
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
220 self.dict[key] = value
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
221 else:
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
222 combined = ", ".join((prev, value))
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
223 self.dict[key] = combined
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
224
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
225 def addcontinue(self, key, more):
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
226 """Add more field data from a continuation line."""
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
227 prev = self.dict[key]
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
228 self.dict[key] = prev + "\n " + more
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
229
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
230 def readheaders(self):
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
231 """Read header lines.
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
232
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
233 Read header lines up to the entirely blank line that terminates them.
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
234 The (normally blank) line that ends the headers is skipped, but not
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
235 included in the returned list. If a non-header line ends the headers,
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
236 (which is an error), an attempt is made to backspace over it; it is
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
237 never included in the returned list.
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
238
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
239 The variable self.status is set to the empty string if all went well,
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
240 otherwise it is an error message. The variable self.headers is a
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
241 completely uninterpreted list of lines contained in the header (so
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
242 printing them will reproduce the header exactly as it appears in the
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
243 file).
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
244
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
245 If multiple header fields with the same name occur, they are combined
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
246 according to the rules in RFC 2616 sec 4.2:
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
247
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
248 Appending each subsequent field-value to the first, each separated
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
249 by a comma. The order in which header fields with the same field-name
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
250 are received is significant to the interpretation of the combined
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
251 field value.
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
252 """
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
253 # XXX The implementation overrides the readheaders() method of
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
254 # rfc822.Message. The base class design isn't amenable to
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
255 # customized behavior here so the method here is a copy of the
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
256 # base class code with a few small changes.
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
257
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
258 self.dict = {}
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
259 self.unixfrom = ''
28083
29965d0d4650 Module review:
Raymond Hettinger <python@rcn.com>
parents: 28063
diff changeset
260 self.headers = hlist = []
24123
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
261 self.status = ''
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
262 headerseen = ""
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
263 firstline = 1
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
264 startofline = unread = tell = None
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
265 if hasattr(self.fp, 'unread'):
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
266 unread = self.fp.unread
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
267 elif self.seekable:
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
268 tell = self.fp.tell
28083
29965d0d4650 Module review:
Raymond Hettinger <python@rcn.com>
parents: 28063
diff changeset
269 while True:
24123
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
270 if tell:
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
271 try:
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
272 startofline = tell()
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
273 except IOError:
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
274 startofline = tell = None
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
275 self.seekable = 0
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
276 line = self.fp.readline()
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
277 if not line:
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
278 self.status = 'EOF in headers'
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
279 break
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
280 # Skip unix From name time lines
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
281 if firstline and line.startswith('From '):
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
282 self.unixfrom = self.unixfrom + line
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
283 continue
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
284 firstline = 0
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
285 if headerseen and line[0] in ' \t':
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
286 # XXX Not sure if continuation lines are handled properly
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
287 # for http and/or for repeating headers
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
288 # It's a continuation line.
28083
29965d0d4650 Module review:
Raymond Hettinger <python@rcn.com>
parents: 28063
diff changeset
289 hlist.append(line)
24123
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
290 self.addcontinue(headerseen, line.strip())
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
291 continue
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
292 elif self.iscomment(line):
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
293 # It's a comment. Ignore it.
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
294 continue
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
295 elif self.islast(line):
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
296 # Note! No pushback here! The delimiter line gets eaten.
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
297 break
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
298 headerseen = self.isheader(line)
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
299 if headerseen:
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
300 # It's a legal header line, save it.
28083
29965d0d4650 Module review:
Raymond Hettinger <python@rcn.com>
parents: 28063
diff changeset
301 hlist.append(line)
24123
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
302 self.addheader(headerseen, line[len(headerseen)+1:].strip())
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
303 continue
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
304 else:
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
305 # It's not a header line; throw it back and stop here.
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
306 if not self.dict:
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
307 self.status = 'No headers'
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
308 else:
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
309 self.status = 'Non-header line where header expected'
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
310 # Try to undo the read.
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
311 if unread:
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
312 unread(line)
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
313 elif tell:
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
314 self.fp.seek(startofline)
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
315 else:
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
316 self.status = self.status + '; bad seek'
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
317 break
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
318
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
319 class HTTPResponse:
24119
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
320
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
321 # strict: If true, raise BadStatusLine if the status line can't be
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
322 # parsed as a valid HTTP/1.0 or 1.1 status line. By default it is
24378
Skip Montanaro <skip@pobox.com>
parents: 24264
diff changeset
323 # false because it prevents clients from talking to HTTP/0.9
24119
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
324 # servers. Note that a response with a sufficiently corrupted
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
325 # status line will look like an HTTP/0.9 response.
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
326
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
327 # See RFC 2616 sec 19.6 and RFC 1945 sec 6 for details.
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
328
50734
d09d6fe31b61 Issue 4879: Allow buffering for HTTPResponse
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 50672
diff changeset
329 def __init__(self, sock, debuglevel=0, strict=0, method=None, buffering=False):
d09d6fe31b61 Issue 4879: Allow buffering for HTTPResponse
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 50672
diff changeset
330 if buffering:
d09d6fe31b61 Issue 4879: Allow buffering for HTTPResponse
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 50672
diff changeset
331 # The caller won't be using any sock.recv() calls, so buffering
55703
e471d6d32724 comment typo fix
Gregory P. Smith <greg@mad-scientist.com>
parents: 55702
diff changeset
332 # is fine and recommended for performance.
50734
d09d6fe31b61 Issue 4879: Allow buffering for HTTPResponse
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 50672
diff changeset
333 self.fp = sock.makefile('rb')
d09d6fe31b61 Issue 4879: Allow buffering for HTTPResponse
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 50672
diff changeset
334 else:
d09d6fe31b61 Issue 4879: Allow buffering for HTTPResponse
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 50672
diff changeset
335 # The buffer size is specified as zero, because the headers of
d09d6fe31b61 Issue 4879: Allow buffering for HTTPResponse
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 50672
diff changeset
336 # the response are read with readline(). If the reads were
d09d6fe31b61 Issue 4879: Allow buffering for HTTPResponse
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 50672
diff changeset
337 # buffered the readline() calls could consume some of the
d09d6fe31b61 Issue 4879: Allow buffering for HTTPResponse
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 50672
diff changeset
338 # response, which make be read via a recv() on the underlying
d09d6fe31b61 Issue 4879: Allow buffering for HTTPResponse
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 50672
diff changeset
339 # socket.
d09d6fe31b61 Issue 4879: Allow buffering for HTTPResponse
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 50672
diff changeset
340 self.fp = sock.makefile('rb', 0)
14672
1b20ad6eb501 Do not close socket when a Content-Length is 0. This make the
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14560
diff changeset
341 self.debuglevel = debuglevel
24119
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
342 self.strict = strict
28978
83fe0f5c9359 SF bug 622042: Don't expect response body from HEAD request.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 28671
diff changeset
343 self._method = method
11527
28b8585354de OpenSSL support. This is based on patches for a version of SSLeay by
Guido van Rossum <guido@python.org>
parents: 7122
diff changeset
344
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
345 self.msg = None
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
346
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
347 # from the Status-Line of the response
15868
3060cbfa01be Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15859
diff changeset
348 self.version = _UNKNOWN # HTTP-Version
3060cbfa01be Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15859
diff changeset
349 self.status = _UNKNOWN # Status-Code
3060cbfa01be Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15859
diff changeset
350 self.reason = _UNKNOWN # Reason-Phrase
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
351
15868
3060cbfa01be Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15859
diff changeset
352 self.chunked = _UNKNOWN # is "chunked" being used?
3060cbfa01be Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15859
diff changeset
353 self.chunk_left = _UNKNOWN # bytes left to read in current chunk
3060cbfa01be Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15859
diff changeset
354 self.length = _UNKNOWN # number of bytes left in response
3060cbfa01be Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15859
diff changeset
355 self.will_close = _UNKNOWN # conn will close at end of response
11527
28b8585354de OpenSSL support. This is based on patches for a version of SSLeay by
Guido van Rossum <guido@python.org>
parents: 7122
diff changeset
356
24038
9377158652dd Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23620
diff changeset
357 def _read_status(self):
24119
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
358 # Initialize with Simple-Response defaults
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
359 line = self.fp.readline()
14672
1b20ad6eb501 Do not close socket when a Content-Length is 0. This make the
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14560
diff changeset
360 if self.debuglevel > 0:
1b20ad6eb501 Do not close socket when a Content-Length is 0. This make the
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14560
diff changeset
361 print "reply:", repr(line)
29688
9fe67f7f3459 Fix sf bug 666219: assertion error in httplib.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 29584
diff changeset
362 if not line:
9fe67f7f3459 Fix sf bug 666219: assertion error in httplib.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 29584
diff changeset
363 # Presumably, the server closed the connection before
9fe67f7f3459 Fix sf bug 666219: assertion error in httplib.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 29584
diff changeset
364 # sending a valid response.
9fe67f7f3459 Fix sf bug 666219: assertion error in httplib.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 29584
diff changeset
365 raise BadStatusLine(line)
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
366 try:
15582
6179d4afa4e4 Get rid of string functions.
Guido van Rossum <guido@python.org>
parents: 15503
diff changeset
367 [version, status, reason] = line.split(None, 2)
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
368 except ValueError:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
369 try:
15582
6179d4afa4e4 Get rid of string functions.
Guido van Rossum <guido@python.org>
parents: 15503
diff changeset
370 [version, status] = line.split(None, 1)
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
371 reason = ""
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
372 except ValueError:
24119
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
373 # empty version will cause next test to fail and status
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
374 # will be treated as 0.9 response.
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
375 version = ""
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
376 if not version.startswith('HTTP/'):
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
377 if self.strict:
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
378 self.close()
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
379 raise BadStatusLine(line)
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
380 else:
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
381 # assume it's a Simple-Response from an 0.9 server
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
382 self.fp = LineAndFileWrapper(line, self.fp)
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
383 return "HTTP/0.9", 200, ""
11527
28b8585354de OpenSSL support. This is based on patches for a version of SSLeay by
Guido van Rossum <guido@python.org>
parents: 7122
diff changeset
384
17482
2265032efc9f SF patch #405845 by Martin von Löwis
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 16400
diff changeset
385 # The status code is a three-digit number
2265032efc9f SF patch #405845 by Martin von Löwis
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 16400
diff changeset
386 try:
24038
9377158652dd Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23620
diff changeset
387 status = int(status)
17482
2265032efc9f SF patch #405845 by Martin von Löwis
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 16400
diff changeset
388 if status < 100 or status > 999:
2265032efc9f SF patch #405845 by Martin von Löwis
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 16400
diff changeset
389 raise BadStatusLine(line)
2265032efc9f SF patch #405845 by Martin von Löwis
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 16400
diff changeset
390 except ValueError:
2265032efc9f SF patch #405845 by Martin von Löwis
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 16400
diff changeset
391 raise BadStatusLine(line)
24038
9377158652dd Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23620
diff changeset
392 return version, status, reason
9377158652dd Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23620
diff changeset
393
24216
c8a19a428358 Change _begin() back to begin().
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24168
diff changeset
394 def begin(self):
24038
9377158652dd Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23620
diff changeset
395 if self.msg is not None:
9377158652dd Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23620
diff changeset
396 # we've already started reading the response
9377158652dd Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23620
diff changeset
397 return
9377158652dd Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23620
diff changeset
398
9377158652dd Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23620
diff changeset
399 # read until we get a non-100 response
28083
29965d0d4650 Module review:
Raymond Hettinger <python@rcn.com>
parents: 28063
diff changeset
400 while True:
24038
9377158652dd Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23620
diff changeset
401 version, status, reason = self._read_status()
33506
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
402 if status != CONTINUE:
24038
9377158652dd Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23620
diff changeset
403 break
9377158652dd Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23620
diff changeset
404 # skip the header from the 100 response
28083
29965d0d4650 Module review:
Raymond Hettinger <python@rcn.com>
parents: 28063
diff changeset
405 while True:
24038
9377158652dd Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23620
diff changeset
406 skip = self.fp.readline().strip()
9377158652dd Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23620
diff changeset
407 if not skip:
9377158652dd Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23620
diff changeset
408 break
9377158652dd Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23620
diff changeset
409 if self.debuglevel > 0:
9377158652dd Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23620
diff changeset
410 print "header:", skip
24263
e6033254df21 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 24262
diff changeset
411
24038
9377158652dd Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23620
diff changeset
412 self.status = status
15582
6179d4afa4e4 Get rid of string functions.
Guido van Rossum <guido@python.org>
parents: 15503
diff changeset
413 self.reason = reason.strip()
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
414 if version == 'HTTP/1.0':
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
415 self.version = 10
15182
b3387eeba68d If the status line is invalid, assume it is a pre-1.0 response. The
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14740
diff changeset
416 elif version.startswith('HTTP/1.'):
15868
3060cbfa01be Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15859
diff changeset
417 self.version = 11 # use HTTP/1.1 code for HTTP/1.x where x>=1
15182
b3387eeba68d If the status line is invalid, assume it is a pre-1.0 response. The
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14740
diff changeset
418 elif version == 'HTTP/0.9':
b3387eeba68d If the status line is invalid, assume it is a pre-1.0 response. The
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14740
diff changeset
419 self.version = 9
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
420 else:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
421 raise UnknownProtocol(version)
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
422
15182
b3387eeba68d If the status line is invalid, assume it is a pre-1.0 response. The
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14740
diff changeset
423 if self.version == 9:
35158
daebfde0e1cf backport bug [ 1155638 ] self.length shield exception in httplib
Georg Brandl <georg@python.org>
parents: 33933
diff changeset
424 self.length = None
24038
9377158652dd Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23620
diff changeset
425 self.chunked = 0
24119
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
426 self.will_close = 1
24123
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
427 self.msg = HTTPMessage(StringIO())
15182
b3387eeba68d If the status line is invalid, assume it is a pre-1.0 response. The
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14740
diff changeset
428 return
b3387eeba68d If the status line is invalid, assume it is a pre-1.0 response. The
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14740
diff changeset
429
24123
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
430 self.msg = HTTPMessage(self.fp, 0)
14672
1b20ad6eb501 Do not close socket when a Content-Length is 0. This make the
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14560
diff changeset
431 if self.debuglevel > 0:
1b20ad6eb501 Do not close socket when a Content-Length is 0. This make the
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14560
diff changeset
432 for hdr in self.msg.headers:
1b20ad6eb501 Do not close socket when a Content-Length is 0. This make the
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14560
diff changeset
433 print "header:", hdr,
1555
6d35308261f4 Towards a standard access mechanism for URLs.
Guido van Rossum <guido@python.org>
parents:
diff changeset
434
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
435 # don't let the msg keep an fp
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
436 self.msg.fp = None
6787
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
437
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
438 # are we using the chunked-style of transfer encoding?
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
439 tr_enc = self.msg.getheader('transfer-encoding')
25107
0ecb3cabf40b SF bug [ 600488 ] Robustness tweak to httplib.py
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24674
diff changeset
440 if tr_enc and tr_enc.lower() == "chunked":
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
441 self.chunked = 1
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
442 self.chunk_left = None
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
443 else:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
444 self.chunked = 0
1555
6d35308261f4 Towards a standard access mechanism for URLs.
Guido van Rossum <guido@python.org>
parents:
diff changeset
445
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
446 # will the connection close at the end of the response?
26187
ae169f5b94e0 Fix SF bug #637789: Handle Proxy-Connection header.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 26126
diff changeset
447 self.will_close = self._check_close()
11527
28b8585354de OpenSSL support. This is based on patches for a version of SSLeay by
Guido van Rossum <guido@python.org>
parents: 7122
diff changeset
448
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
449 # do we have a Content-Length?
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
450 # NOTE: RFC 2616, S4.4, #3 says we ignore this if tr_enc is "chunked"
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
451 length = self.msg.getheader('content-length')
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
452 if length and not self.chunked:
14560
eeb8ee488106 cope with weird Content-Length values returned from servers by
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14140
diff changeset
453 try:
eeb8ee488106 cope with weird Content-Length values returned from servers by
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14140
diff changeset
454 self.length = int(length)
eeb8ee488106 cope with weird Content-Length values returned from servers by
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14140
diff changeset
455 except ValueError:
eeb8ee488106 cope with weird Content-Length values returned from servers by
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14140
diff changeset
456 self.length = None
45494
7efa4d1bced3 #1627: httplib now ignores negative Content-Length headers.
Georg Brandl <georg@python.org>
parents: 45493
diff changeset
457 else:
7efa4d1bced3 #1627: httplib now ignores negative Content-Length headers.
Georg Brandl <georg@python.org>
parents: 45493
diff changeset
458 if self.length < 0: # ignore nonsensical negative lengths
7efa4d1bced3 #1627: httplib now ignores negative Content-Length headers.
Georg Brandl <georg@python.org>
parents: 45493
diff changeset
459 self.length = None
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
460 else:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
461 self.length = None
11527
28b8585354de OpenSSL support. This is based on patches for a version of SSLeay by
Guido van Rossum <guido@python.org>
parents: 7122
diff changeset
462
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
463 # does the body have a fixed length? (of zero)
33506
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
464 if (status == NO_CONTENT or status == NOT_MODIFIED or
28978
83fe0f5c9359 SF bug 622042: Don't expect response body from HEAD request.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 28671
diff changeset
465 100 <= status < 200 or # 1xx codes
83fe0f5c9359 SF bug 622042: Don't expect response body from HEAD request.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 28671
diff changeset
466 self._method == 'HEAD'):
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
467 self.length = 0
1555
6d35308261f4 Towards a standard access mechanism for URLs.
Guido van Rossum <guido@python.org>
parents:
diff changeset
468
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
469 # if the connection remains open, and we aren't using chunked, and
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
470 # a content-length was not provided, then assume that the connection
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
471 # WILL close.
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
472 if not self.will_close and \
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
473 not self.chunked and \
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
474 self.length is None:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
475 self.will_close = 1
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
476
26187
ae169f5b94e0 Fix SF bug #637789: Handle Proxy-Connection header.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 26126
diff changeset
477 def _check_close(self):
33001
4fd59651fe3f SF bug 874842 and patch 997626: httplib bugs
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 32252
diff changeset
478 conn = self.msg.getheader('connection')
26187
ae169f5b94e0 Fix SF bug #637789: Handle Proxy-Connection header.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 26126
diff changeset
479 if self.version == 11:
ae169f5b94e0 Fix SF bug #637789: Handle Proxy-Connection header.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 26126
diff changeset
480 # An HTTP/1.1 proxy is assumed to stay open unless
ae169f5b94e0 Fix SF bug #637789: Handle Proxy-Connection header.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 26126
diff changeset
481 # explicitly closed.
ae169f5b94e0 Fix SF bug #637789: Handle Proxy-Connection header.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 26126
diff changeset
482 conn = self.msg.getheader('connection')
31967
3d046d063a84 Replace str.find()!=1 with the more readable "in" operator.
Raymond Hettinger <python@rcn.com>
parents: 31389
diff changeset
483 if conn and "close" in conn.lower():
26187
ae169f5b94e0 Fix SF bug #637789: Handle Proxy-Connection header.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 26126
diff changeset
484 return True
ae169f5b94e0 Fix SF bug #637789: Handle Proxy-Connection header.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 26126
diff changeset
485 return False
ae169f5b94e0 Fix SF bug #637789: Handle Proxy-Connection header.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 26126
diff changeset
486
33001
4fd59651fe3f SF bug 874842 and patch 997626: httplib bugs
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 32252
diff changeset
487 # Some HTTP/1.0 implementations have support for persistent
4fd59651fe3f SF bug 874842 and patch 997626: httplib bugs
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 32252
diff changeset
488 # connections, using rules different than HTTP/1.1.
26187
ae169f5b94e0 Fix SF bug #637789: Handle Proxy-Connection header.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 26126
diff changeset
489
44218
77cb8ff34912 Fix tpyo.
Georg Brandl <georg@python.org>
parents: 43584
diff changeset
490 # For older HTTP, Keep-Alive indicates persistent connection.
26187
ae169f5b94e0 Fix SF bug #637789: Handle Proxy-Connection header.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 26126
diff changeset
491 if self.msg.getheader('keep-alive'):
ae169f5b94e0 Fix SF bug #637789: Handle Proxy-Connection header.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 26126
diff changeset
492 return False
26387
fcf9734e1021 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 26188
diff changeset
493
33001
4fd59651fe3f SF bug 874842 and patch 997626: httplib bugs
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 32252
diff changeset
494 # At least Akamai returns a "Connection: Keep-Alive" header,
4fd59651fe3f SF bug 874842 and patch 997626: httplib bugs
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 32252
diff changeset
495 # which was supposed to be sent by the client.
4fd59651fe3f SF bug 874842 and patch 997626: httplib bugs
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 32252
diff changeset
496 if conn and "keep-alive" in conn.lower():
4fd59651fe3f SF bug 874842 and patch 997626: httplib bugs
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 32252
diff changeset
497 return False
4fd59651fe3f SF bug 874842 and patch 997626: httplib bugs
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 32252
diff changeset
498
26187
ae169f5b94e0 Fix SF bug #637789: Handle Proxy-Connection header.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 26126
diff changeset
499 # Proxy-Connection is a netscape hack.
ae169f5b94e0 Fix SF bug #637789: Handle Proxy-Connection header.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 26126
diff changeset
500 pconn = self.msg.getheader('proxy-connection')
31967
3d046d063a84 Replace str.find()!=1 with the more readable "in" operator.
Raymond Hettinger <python@rcn.com>
parents: 31389
diff changeset
501 if pconn and "keep-alive" in pconn.lower():
26187
ae169f5b94e0 Fix SF bug #637789: Handle Proxy-Connection header.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 26126
diff changeset
502 return False
ae169f5b94e0 Fix SF bug #637789: Handle Proxy-Connection header.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 26126
diff changeset
503
ae169f5b94e0 Fix SF bug #637789: Handle Proxy-Connection header.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 26126
diff changeset
504 # otherwise, assume it will close
ae169f5b94e0 Fix SF bug #637789: Handle Proxy-Connection header.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 26126
diff changeset
505 return True
ae169f5b94e0 Fix SF bug #637789: Handle Proxy-Connection header.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 26126
diff changeset
506
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
507 def close(self):
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
508 if self.fp:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
509 self.fp.close()
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
510 self.fp = None
11527
28b8585354de OpenSSL support. This is based on patches for a version of SSLeay by
Guido van Rossum <guido@python.org>
parents: 7122
diff changeset
511
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
512 def isclosed(self):
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
513 # NOTE: it is possible that we will not ever call self.close(). This
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
514 # case occurs when will_close is TRUE, length is None, and we
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
515 # read up to the last byte, but NOT past it.
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
516 #
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
517 # IMPLIES: if will_close is FALSE, then self.close() will ALWAYS be
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
518 # called, meaning self.isclosed() is meaningful.
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
519 return self.fp is None
6787
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
520
33001
4fd59651fe3f SF bug 874842 and patch 997626: httplib bugs
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 32252
diff changeset
521 # XXX It would be nice to have readline and __iter__ for this, too.
4fd59651fe3f SF bug 874842 and patch 997626: httplib bugs
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 32252
diff changeset
522
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
523 def read(self, amt=None):
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
524 if self.fp is None:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
525 return ''
1555
6d35308261f4 Towards a standard access mechanism for URLs.
Guido van Rossum <guido@python.org>
parents:
diff changeset
526
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
527 if self.chunked:
25112
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
528 return self._read_chunked(amt)
26126
98f3fd1ed42f Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 25112
diff changeset
529
25112
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
530 if amt is None:
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
531 # unbounded read
33933
eb231544c1ad Fix for SF bug 988120 via patch 1061941.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 33761
diff changeset
532 if self.length is None:
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
533 s = self.fp.read()
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
534 else:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
535 s = self._safe_read(self.length)
33933
eb231544c1ad Fix for SF bug 988120 via patch 1061941.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 33761
diff changeset
536 self.length = 0
15868
3060cbfa01be Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15859
diff changeset
537 self.close() # we read everything
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
538 return s
1555
6d35308261f4 Towards a standard access mechanism for URLs.
Guido van Rossum <guido@python.org>
parents:
diff changeset
539
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
540 if self.length is not None:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
541 if amt > self.length:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
542 # clip the read to the "end of response"
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
543 amt = self.length
1555
6d35308261f4 Towards a standard access mechanism for URLs.
Guido van Rossum <guido@python.org>
parents:
diff changeset
544
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
545 # we do not use _safe_read() here because this may be a .will_close
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
546 # connection, and the user is reading more bytes than will be provided
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
547 # (for example, reading in 1k chunks)
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
548 s = self.fp.read(amt)
33933
eb231544c1ad Fix for SF bug 988120 via patch 1061941.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 33761
diff changeset
549 if self.length is not None:
eb231544c1ad Fix for SF bug 988120 via patch 1061941.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 33761
diff changeset
550 self.length -= len(s)
43584
da44334ebea2 Issue #1580738. When HTTPConnection reads the whole stream with read(),
Facundo Batista <facundobatista@gmail.com>
parents: 43325
diff changeset
551 if not self.length:
da44334ebea2 Issue #1580738. When HTTPConnection reads the whole stream with read(),
Facundo Batista <facundobatista@gmail.com>
parents: 43325
diff changeset
552 self.close()
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
553 return s
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
554
25112
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
555 def _read_chunked(self, amt):
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
556 assert self.chunked != _UNKNOWN
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
557 chunk_left = self.chunk_left
55827
77db25c7526a Fixes issue #6838: use a list to accumulate the value instead of repeatedly concatenating strings.
Chris Withers <chris@simplistix.co.uk>
parents: 55703
diff changeset
558 value = []
28083
29965d0d4650 Module review:
Raymond Hettinger <python@rcn.com>
parents: 28063
diff changeset
559 while True:
25112
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
560 if chunk_left is None:
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
561 line = self.fp.readline()
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
562 i = line.find(';')
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
563 if i >= 0:
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
564 line = line[:i] # strip chunk-extensions
45493
21a5e9a5483a #900744: If an invalid chunked-encoding header is sent by a server,
Georg Brandl <georg@python.org>
parents: 45468
diff changeset
565 try:
21a5e9a5483a #900744: If an invalid chunked-encoding header is sent by a server,
Georg Brandl <georg@python.org>
parents: 45468
diff changeset
566 chunk_left = int(line, 16)
21a5e9a5483a #900744: If an invalid chunked-encoding header is sent by a server,
Georg Brandl <georg@python.org>
parents: 45468
diff changeset
567 except ValueError:
21a5e9a5483a #900744: If an invalid chunked-encoding header is sent by a server,
Georg Brandl <georg@python.org>
parents: 45468
diff changeset
568 # close the connection as protocol synchronisation is
21a5e9a5483a #900744: If an invalid chunked-encoding header is sent by a server,
Georg Brandl <georg@python.org>
parents: 45468
diff changeset
569 # probably lost
21a5e9a5483a #900744: If an invalid chunked-encoding header is sent by a server,
Georg Brandl <georg@python.org>
parents: 45468
diff changeset
570 self.close()
55827
77db25c7526a Fixes issue #6838: use a list to accumulate the value instead of repeatedly concatenating strings.
Chris Withers <chris@simplistix.co.uk>
parents: 55703
diff changeset
571 raise IncompleteRead(''.join(value))
25112
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
572 if chunk_left == 0:
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
573 break
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
574 if amt is None:
55827
77db25c7526a Fixes issue #6838: use a list to accumulate the value instead of repeatedly concatenating strings.
Chris Withers <chris@simplistix.co.uk>
parents: 55703
diff changeset
575 value.append(self._safe_read(chunk_left))
25112
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
576 elif amt < chunk_left:
55827
77db25c7526a Fixes issue #6838: use a list to accumulate the value instead of repeatedly concatenating strings.
Chris Withers <chris@simplistix.co.uk>
parents: 55703
diff changeset
577 value.append(self._safe_read(amt))
25112
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
578 self.chunk_left = chunk_left - amt
55827
77db25c7526a Fixes issue #6838: use a list to accumulate the value instead of repeatedly concatenating strings.
Chris Withers <chris@simplistix.co.uk>
parents: 55703
diff changeset
579 return ''.join(value)
25112
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
580 elif amt == chunk_left:
55827
77db25c7526a Fixes issue #6838: use a list to accumulate the value instead of repeatedly concatenating strings.
Chris Withers <chris@simplistix.co.uk>
parents: 55703
diff changeset
581 value.append(self._safe_read(amt))
25112
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
582 self._safe_read(2) # toss the CRLF at the end of the chunk
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
583 self.chunk_left = None
55827
77db25c7526a Fixes issue #6838: use a list to accumulate the value instead of repeatedly concatenating strings.
Chris Withers <chris@simplistix.co.uk>
parents: 55703
diff changeset
584 return ''.join(value)
25112
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
585 else:
55827
77db25c7526a Fixes issue #6838: use a list to accumulate the value instead of repeatedly concatenating strings.
Chris Withers <chris@simplistix.co.uk>
parents: 55703
diff changeset
586 value.append(self._safe_read(chunk_left))
25112
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
587 amt -= chunk_left
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
588
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
589 # we read the whole chunk, get another
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
590 self._safe_read(2) # toss the CRLF at the end of the chunk
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
591 chunk_left = None
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
592
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
593 # read and discard trailer up to the CRLF terminator
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
594 ### note: we shouldn't have any trailers!
28083
29965d0d4650 Module review:
Raymond Hettinger <python@rcn.com>
parents: 28063
diff changeset
595 while True:
25112
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
596 line = self.fp.readline()
45298
8eb8e9a3260d Patch #1966: Break infinite loop in httplib when the servers
Martin v. Löwis <martin@v.loewis.de>
parents: 44218
diff changeset
597 if not line:
8eb8e9a3260d Patch #1966: Break infinite loop in httplib when the servers
Martin v. Löwis <martin@v.loewis.de>
parents: 44218
diff changeset
598 # a vanishingly small number of sites EOF without
8eb8e9a3260d Patch #1966: Break infinite loop in httplib when the servers
Martin v. Löwis <martin@v.loewis.de>
parents: 44218
diff changeset
599 # sending the trailer
8eb8e9a3260d Patch #1966: Break infinite loop in httplib when the servers
Martin v. Löwis <martin@v.loewis.de>
parents: 44218
diff changeset
600 break
25112
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
601 if line == '\r\n':
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
602 break
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
603
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
604 # we read everything; close the "file"
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
605 self.close()
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
606
55827
77db25c7526a Fixes issue #6838: use a list to accumulate the value instead of repeatedly concatenating strings.
Chris Withers <chris@simplistix.co.uk>
parents: 55703
diff changeset
607 return ''.join(value)
26126
98f3fd1ed42f Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 25112
diff changeset
608
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
609 def _safe_read(self, amt):
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
610 """Read the number of bytes requested, compensating for partial reads.
11527
28b8585354de OpenSSL support. This is based on patches for a version of SSLeay by
Guido van Rossum <guido@python.org>
parents: 7122
diff changeset
611
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
612 Normally, we have a blocking socket, but a read() can be interrupted
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
613 by a signal (resulting in a partial read).
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
614
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
615 Note that we cannot distinguish between EOF and an interrupt when zero
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
616 bytes have been read. IncompleteRead() will be raised in this
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
617 situation.
11527
28b8585354de OpenSSL support. This is based on patches for a version of SSLeay by
Guido van Rossum <guido@python.org>
parents: 7122
diff changeset
618
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
619 This function should be used when <amt> bytes "should" be present for
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
620 reading. If the bytes are truly not available (due to EOF), then the
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
621 IncompleteRead exception can be used to detect the problem.
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
622 """
55650
1377a8c26792 Force the http connection to close after any request returned when
Gregory P. Smith <greg@mad-scientist.com>
parents: 54298
diff changeset
623 # NOTE(gps): As of svn r74426 socket._fileobject.read(x) will never
1377a8c26792 Force the http connection to close after any request returned when
Gregory P. Smith <greg@mad-scientist.com>
parents: 54298
diff changeset
624 # return less than x bytes unless EOF is encountered. It now handles
1377a8c26792 Force the http connection to close after any request returned when
Gregory P. Smith <greg@mad-scientist.com>
parents: 54298
diff changeset
625 # signal interruptions (socket.error EINTR) internally. This code
1377a8c26792 Force the http connection to close after any request returned when
Gregory P. Smith <greg@mad-scientist.com>
parents: 54298
diff changeset
626 # never caught that exception anyways. It seems largely pointless.
1377a8c26792 Force the http connection to close after any request returned when
Gregory P. Smith <greg@mad-scientist.com>
parents: 54298
diff changeset
627 # self.fp.read(amt) will work fine.
35652
6a20831d357d bug [ 1296004 ] MemoryError in httplib
Georg Brandl <georg@python.org>
parents: 35158
diff changeset
628 s = []
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
629 while amt > 0:
35652
6a20831d357d bug [ 1296004 ] MemoryError in httplib
Georg Brandl <georg@python.org>
parents: 35158
diff changeset
630 chunk = self.fp.read(min(amt, MAXAMOUNT))
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
631 if not chunk:
51951
87115c727102 give httplib.IncompleteRead a more sane repr #4308
Benjamin Peterson <benjamin@python.org>
parents: 50734
diff changeset
632 raise IncompleteRead(''.join(s), amt)
35652
6a20831d357d bug [ 1296004 ] MemoryError in httplib
Georg Brandl <georg@python.org>
parents: 35158
diff changeset
633 s.append(chunk)
28083
29965d0d4650 Module review:
Raymond Hettinger <python@rcn.com>
parents: 28063
diff changeset
634 amt -= len(chunk)
35652
6a20831d357d bug [ 1296004 ] MemoryError in httplib
Georg Brandl <georg@python.org>
parents: 35158
diff changeset
635 return ''.join(s)
11527
28b8585354de OpenSSL support. This is based on patches for a version of SSLeay by
Guido van Rossum <guido@python.org>
parents: 7122
diff changeset
636
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
637 def getheader(self, name, default=None):
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
638 if self.msg is None:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
639 raise ResponseNotReady()
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
640 return self.msg.getheader(name, default)
11527
28b8585354de OpenSSL support. This is based on patches for a version of SSLeay by
Guido van Rossum <guido@python.org>
parents: 7122
diff changeset
641
33173
e05e9d05830d Patch #800236: add HTTPResponse.getheaders().
Martin v. Löwis <martin@v.loewis.de>
parents: 33001
diff changeset
642 def getheaders(self):
e05e9d05830d Patch #800236: add HTTPResponse.getheaders().
Martin v. Löwis <martin@v.loewis.de>
parents: 33001
diff changeset
643 """Return list of (header, value) tuples."""
e05e9d05830d Patch #800236: add HTTPResponse.getheaders().
Martin v. Löwis <martin@v.loewis.de>
parents: 33001
diff changeset
644 if self.msg is None:
e05e9d05830d Patch #800236: add HTTPResponse.getheaders().
Martin v. Löwis <martin@v.loewis.de>
parents: 33001
diff changeset
645 raise ResponseNotReady()
e05e9d05830d Patch #800236: add HTTPResponse.getheaders().
Martin v. Löwis <martin@v.loewis.de>
parents: 33001
diff changeset
646 return self.msg.items()
e05e9d05830d Patch #800236: add HTTPResponse.getheaders().
Martin v. Löwis <martin@v.loewis.de>
parents: 33001
diff changeset
647
1555
6d35308261f4 Towards a standard access mechanism for URLs.
Guido van Rossum <guido@python.org>
parents:
diff changeset
648
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
649 class HTTPConnection:
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
650
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
651 _http_vsn = 11
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
652 _http_vsn_str = 'HTTP/1.1'
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
653
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
654 response_class = HTTPResponse
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
655 default_port = HTTP_PORT
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
656 auto_open = 1
14672
1b20ad6eb501 Do not close socket when a Content-Length is 0. This make the
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14560
diff changeset
657 debuglevel = 0
24119
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
658 strict = 0
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
659
47372
e2c5ae30cffe Fixed the semantic of timeout for socket.create_connection and
Facundo Batista <facundobatista@gmail.com>
parents: 45494
diff changeset
660 def __init__(self, host, port=None, strict=None,
58113
a19ad299095f issue3972: HTTPConnection and HTTPSConnection now support a
Gregory P. Smith <greg@mad-scientist.com>
parents: 57799
diff changeset
661 timeout=socket._GLOBAL_DEFAULT_TIMEOUT, source_address=None):
41288
a73aa0b821ca Added a 'create_connect()' function to socket.py, which creates a
Facundo Batista <facundobatista@gmail.com>
parents: 41285
diff changeset
662 self.timeout = timeout
58113
a19ad299095f issue3972: HTTPConnection and HTTPSConnection now support a
Gregory P. Smith <greg@mad-scientist.com>
parents: 57799
diff changeset
663 self.source_address = source_address
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
664 self.sock = None
24262
3b571e32a6e4 Send HTTP requests with a single send() call instead of many.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24216
diff changeset
665 self._buffer = []
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
666 self.__response = None
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
667 self.__state = _CS_IDLE
28978
83fe0f5c9359 SF bug 622042: Don't expect response body from HEAD request.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 28671
diff changeset
668 self._method = None
54298
e7b4a443f764 Fixed Issue1424152, urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 51951
diff changeset
669 self._tunnel_host = None
e7b4a443f764 Fixed Issue1424152, urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 51951
diff changeset
670 self._tunnel_port = None
57799
10970f64ba33 Fix for issue 7291 - urllib2 cannot handle https with proxy requiring auth
Senthil Kumaran <orsenthil@gmail.com>
parents: 56528
diff changeset
671 self._tunnel_headers = {}
24263
e6033254df21 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 24262
diff changeset
672
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
673 self._set_hostport(host, port)
24119
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
674 if strict is not None:
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
675 self.strict = strict
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
676
57799
10970f64ba33 Fix for issue 7291 - urllib2 cannot handle https with proxy requiring auth
Senthil Kumaran <orsenthil@gmail.com>
parents: 56528
diff changeset
677 def set_tunnel(self, host, port=None, headers=None):
10970f64ba33 Fix for issue 7291 - urllib2 cannot handle https with proxy requiring auth
Senthil Kumaran <orsenthil@gmail.com>
parents: 56528
diff changeset
678 """ Sets up the host and the port for the HTTP CONNECT Tunnelling.
10970f64ba33 Fix for issue 7291 - urllib2 cannot handle https with proxy requiring auth
Senthil Kumaran <orsenthil@gmail.com>
parents: 56528
diff changeset
679
10970f64ba33 Fix for issue 7291 - urllib2 cannot handle https with proxy requiring auth
Senthil Kumaran <orsenthil@gmail.com>
parents: 56528
diff changeset
680 The headers argument should be a mapping of extra HTTP headers
10970f64ba33 Fix for issue 7291 - urllib2 cannot handle https with proxy requiring auth
Senthil Kumaran <orsenthil@gmail.com>
parents: 56528
diff changeset
681 to send with the CONNECT request.
10970f64ba33 Fix for issue 7291 - urllib2 cannot handle https with proxy requiring auth
Senthil Kumaran <orsenthil@gmail.com>
parents: 56528
diff changeset
682 """
54298
e7b4a443f764 Fixed Issue1424152, urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 51951
diff changeset
683 self._tunnel_host = host
e7b4a443f764 Fixed Issue1424152, urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 51951
diff changeset
684 self._tunnel_port = port
57799
10970f64ba33 Fix for issue 7291 - urllib2 cannot handle https with proxy requiring auth
Senthil Kumaran <orsenthil@gmail.com>
parents: 56528
diff changeset
685 if headers:
10970f64ba33 Fix for issue 7291 - urllib2 cannot handle https with proxy requiring auth
Senthil Kumaran <orsenthil@gmail.com>
parents: 56528
diff changeset
686 self._tunnel_headers = headers
10970f64ba33 Fix for issue 7291 - urllib2 cannot handle https with proxy requiring auth
Senthil Kumaran <orsenthil@gmail.com>
parents: 56528
diff changeset
687 else:
10970f64ba33 Fix for issue 7291 - urllib2 cannot handle https with proxy requiring auth
Senthil Kumaran <orsenthil@gmail.com>
parents: 56528
diff changeset
688 self._tunnel_headers.clear()
54298
e7b4a443f764 Fixed Issue1424152, urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 51951
diff changeset
689
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
690 def _set_hostport(self, host, port):
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
691 if port is None:
33488
71ed4b35fdc4 Search from the end of the host/port combination to find the colon which
Skip Montanaro <skip@pobox.com>
parents: 33173
diff changeset
692 i = host.rfind(':')
33491
0c299cabf4f0 missed the obvious test case and corresponding fix
Skip Montanaro <skip@pobox.com>
parents: 33488
diff changeset
693 j = host.rfind(']') # ipv6 addresses have [...]
0c299cabf4f0 missed the obvious test case and corresponding fix
Skip Montanaro <skip@pobox.com>
parents: 33488
diff changeset
694 if i > j:
22494
7224a1df4809 add InvalidURL exception - raised if port is given but empty or non-numeric
Skip Montanaro <skip@pobox.com>
parents: 22403
diff changeset
695 try:
7224a1df4809 add InvalidURL exception - raised if port is given but empty or non-numeric
Skip Montanaro <skip@pobox.com>
parents: 22403
diff changeset
696 port = int(host[i+1:])
7224a1df4809 add InvalidURL exception - raised if port is given but empty or non-numeric
Skip Montanaro <skip@pobox.com>
parents: 22403
diff changeset
697 except ValueError:
24088
b74d4a59793a Convert raise to call exception class. Add whitespace.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24042
diff changeset
698 raise InvalidURL("nonnumeric port: '%s'" % host[i+1:])
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
699 host = host[:i]
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
700 else:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
701 port = self.default_port
33761
d1228127d294 SF bug #1046855: httplib index out of range
Raymond Hettinger <python@rcn.com>
parents: 33506
diff changeset
702 if host and host[0] == '[' and host[-1] == ']':
33496
9e2b94a3b5dc Remove tabs.
Brett Cannon <bcannon@gmail.com>
parents: 33493
diff changeset
703 host = host[1:-1]
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
704 self.host = host
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
705 self.port = port
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
706
14672
1b20ad6eb501 Do not close socket when a Content-Length is 0. This make the
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14560
diff changeset
707 def set_debuglevel(self, level):
1b20ad6eb501 Do not close socket when a Content-Length is 0. This make the
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14560
diff changeset
708 self.debuglevel = level
1b20ad6eb501 Do not close socket when a Content-Length is 0. This make the
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14560
diff changeset
709
54298
e7b4a443f764 Fixed Issue1424152, urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 51951
diff changeset
710 def _tunnel(self):
e7b4a443f764 Fixed Issue1424152, urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 51951
diff changeset
711 self._set_hostport(self._tunnel_host, self._tunnel_port)
57799
10970f64ba33 Fix for issue 7291 - urllib2 cannot handle https with proxy requiring auth
Senthil Kumaran <orsenthil@gmail.com>
parents: 56528
diff changeset
712 self.send("CONNECT %s:%d HTTP/1.0\r\n" % (self.host, self.port))
10970f64ba33 Fix for issue 7291 - urllib2 cannot handle https with proxy requiring auth
Senthil Kumaran <orsenthil@gmail.com>
parents: 56528
diff changeset
713 for header, value in self._tunnel_headers.iteritems():
10970f64ba33 Fix for issue 7291 - urllib2 cannot handle https with proxy requiring auth
Senthil Kumaran <orsenthil@gmail.com>
parents: 56528
diff changeset
714 self.send("%s: %s\r\n" % (header, value))
10970f64ba33 Fix for issue 7291 - urllib2 cannot handle https with proxy requiring auth
Senthil Kumaran <orsenthil@gmail.com>
parents: 56528
diff changeset
715 self.send("\r\n")
54298
e7b4a443f764 Fixed Issue1424152, urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 51951
diff changeset
716 response = self.response_class(self.sock, strict = self.strict,
e7b4a443f764 Fixed Issue1424152, urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 51951
diff changeset
717 method = self._method)
e7b4a443f764 Fixed Issue1424152, urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 51951
diff changeset
718 (version, code, message) = response._read_status()
e7b4a443f764 Fixed Issue1424152, urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 51951
diff changeset
719
e7b4a443f764 Fixed Issue1424152, urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 51951
diff changeset
720 if code != 200:
e7b4a443f764 Fixed Issue1424152, urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 51951
diff changeset
721 self.close()
56528
64e19a689886 Fix py3k warnings in httplib
Antoine Pitrou <solipsis@pitrou.net>
parents: 56285
diff changeset
722 raise socket.error("Tunnel connection failed: %d %s" % (code,
64e19a689886 Fix py3k warnings in httplib
Antoine Pitrou <solipsis@pitrou.net>
parents: 56285
diff changeset
723 message.strip()))
54298
e7b4a443f764 Fixed Issue1424152, urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 51951
diff changeset
724 while True:
e7b4a443f764 Fixed Issue1424152, urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 51951
diff changeset
725 line = response.fp.readline()
e7b4a443f764 Fixed Issue1424152, urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 51951
diff changeset
726 if line == '\r\n': break
e7b4a443f764 Fixed Issue1424152, urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 51951
diff changeset
727
e7b4a443f764 Fixed Issue1424152, urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 51951
diff changeset
728
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
729 def connect(self):
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
730 """Connect to the host and port specified in __init__."""
41305
8be744d5c567 Some nits.
Georg Brandl <georg@python.org>
parents: 41288
diff changeset
731 self.sock = socket.create_connection((self.host,self.port),
58113
a19ad299095f issue3972: HTTPConnection and HTTPSConnection now support a
Gregory P. Smith <greg@mad-scientist.com>
parents: 57799
diff changeset
732 self.timeout, self.source_address)
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
733
54298
e7b4a443f764 Fixed Issue1424152, urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 51951
diff changeset
734 if self._tunnel_host:
e7b4a443f764 Fixed Issue1424152, urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 51951
diff changeset
735 self._tunnel()
e7b4a443f764 Fixed Issue1424152, urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 51951
diff changeset
736
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
737 def close(self):
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
738 """Close the connection to the HTTP server."""
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
739 if self.sock:
15868
3060cbfa01be Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15859
diff changeset
740 self.sock.close() # close it manually... there may be other refs
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
741 self.sock = None
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
742 if self.__response:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
743 self.__response.close()
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
744 self.__response = None
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
745 self.__state = _CS_IDLE
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
746
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
747 def send(self, str):
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
748 """Send `str' to the server."""
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
749 if self.sock is None:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
750 if self.auto_open:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
751 self.connect()
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
752 else:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
753 raise NotConnected()
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
754
24264
18c1b290e484 Fix typos and such caught by the pycheckerbot.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24263
diff changeset
755 # send the data to the server. if we get a broken pipe, then close
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
756 # the socket. we want to reconnect when somebody tries to send again.
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
757 #
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
758 # NOTE: we DO propagate the error, though, because we cannot simply
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
759 # ignore the error... the caller will know if they can retry.
14672
1b20ad6eb501 Do not close socket when a Content-Length is 0. This make the
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14560
diff changeset
760 if self.debuglevel > 0:
1b20ad6eb501 Do not close socket when a Content-Length is 0. This make the
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14560
diff changeset
761 print "send:", repr(str)
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
762 try:
40333
90d355829bbb Patch #1065257: Support passing open files as body in
Martin v. Löwis <martin@v.loewis.de>
parents: 39293
diff changeset
763 blocksize=8192
56285
e0f234116ab1 Issue #6790: Make it possible again to pass an `array.array` to
Antoine Pitrou <solipsis@pitrou.net>
parents: 55827
diff changeset
764 if hasattr(str,'read') and not isinstance(str, array):
40333
90d355829bbb Patch #1065257: Support passing open files as body in
Martin v. Löwis <martin@v.loewis.de>
parents: 39293
diff changeset
765 if self.debuglevel > 0: print "sendIng a read()able"
90d355829bbb Patch #1065257: Support passing open files as body in
Martin v. Löwis <martin@v.loewis.de>
parents: 39293
diff changeset
766 data=str.read(blocksize)
90d355829bbb Patch #1065257: Support passing open files as body in
Martin v. Löwis <martin@v.loewis.de>
parents: 39293
diff changeset
767 while data:
90d355829bbb Patch #1065257: Support passing open files as body in
Martin v. Löwis <martin@v.loewis.de>
parents: 39293
diff changeset
768 self.sock.sendall(data)
90d355829bbb Patch #1065257: Support passing open files as body in
Martin v. Löwis <martin@v.loewis.de>
parents: 39293
diff changeset
769 data=str.read(blocksize)
90d355829bbb Patch #1065257: Support passing open files as body in
Martin v. Löwis <martin@v.loewis.de>
parents: 39293
diff changeset
770 else:
90d355829bbb Patch #1065257: Support passing open files as body in
Martin v. Löwis <martin@v.loewis.de>
parents: 39293
diff changeset
771 self.sock.sendall(str)
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
772 except socket.error, v:
56528
64e19a689886 Fix py3k warnings in httplib
Antoine Pitrou <solipsis@pitrou.net>
parents: 56285
diff changeset
773 if v.args[0] == 32: # Broken pipe
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
774 self.close()
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
775 raise
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
776
24262
3b571e32a6e4 Send HTTP requests with a single send() call instead of many.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24216
diff changeset
777 def _output(self, s):
3b571e32a6e4 Send HTTP requests with a single send() call instead of many.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24216
diff changeset
778 """Add a line of output to the current request buffer.
24674
3269bdd65988 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 24378
diff changeset
779
24264
18c1b290e484 Fix typos and such caught by the pycheckerbot.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24263
diff changeset
780 Assumes that the line does *not* end with \\r\\n.
24262
3b571e32a6e4 Send HTTP requests with a single send() call instead of many.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24216
diff changeset
781 """
3b571e32a6e4 Send HTTP requests with a single send() call instead of many.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24216
diff changeset
782 self._buffer.append(s)
3b571e32a6e4 Send HTTP requests with a single send() call instead of many.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24216
diff changeset
783
49890
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
784 def _send_output(self, message_body=None):
24262
3b571e32a6e4 Send HTTP requests with a single send() call instead of many.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24216
diff changeset
785 """Send the currently buffered request and clear the buffer.
3b571e32a6e4 Send HTTP requests with a single send() call instead of many.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24216
diff changeset
786
24264
18c1b290e484 Fix typos and such caught by the pycheckerbot.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24263
diff changeset
787 Appends an extra \\r\\n to the buffer.
50672
07bbc2e20aa4 Issue 4336: HTTPRequest._send_output() now deals with the case of the message body not being a string. This allows clients to use endheaders(message_body) instead of endheaders() + send(message_body) without making any extra checks.
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 49890
diff changeset
788 A message_body may be specified, to be appended to the request.
24262
3b571e32a6e4 Send HTTP requests with a single send() call instead of many.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24216
diff changeset
789 """
3b571e32a6e4 Send HTTP requests with a single send() call instead of many.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24216
diff changeset
790 self._buffer.extend(("", ""))
3b571e32a6e4 Send HTTP requests with a single send() call instead of many.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24216
diff changeset
791 msg = "\r\n".join(self._buffer)
3b571e32a6e4 Send HTTP requests with a single send() call instead of many.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24216
diff changeset
792 del self._buffer[:]
49890
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
793 # If msg and message_body are sent in a single send() call,
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
794 # it will avoid performance problems caused by the interaction
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
795 # between delayed ack and the Nagle algorithim.
50672
07bbc2e20aa4 Issue 4336: HTTPRequest._send_output() now deals with the case of the message body not being a string. This allows clients to use endheaders(message_body) instead of endheaders() + send(message_body) without making any extra checks.
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 49890
diff changeset
796 if isinstance(message_body, str):
49890
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
797 msg += message_body
50672
07bbc2e20aa4 Issue 4336: HTTPRequest._send_output() now deals with the case of the message body not being a string. This allows clients to use endheaders(message_body) instead of endheaders() + send(message_body) without making any extra checks.
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 49890
diff changeset
798 message_body = None
24262
3b571e32a6e4 Send HTTP requests with a single send() call instead of many.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24216
diff changeset
799 self.send(msg)
50672
07bbc2e20aa4 Issue 4336: HTTPRequest._send_output() now deals with the case of the message body not being a string. This allows clients to use endheaders(message_body) instead of endheaders() + send(message_body) without making any extra checks.
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 49890
diff changeset
800 if message_body is not None:
07bbc2e20aa4 Issue 4336: HTTPRequest._send_output() now deals with the case of the message body not being a string. This allows clients to use endheaders(message_body) instead of endheaders() + send(message_body) without making any extra checks.
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 49890
diff changeset
801 #message_body was not a string (i.e. it is a file) and
07bbc2e20aa4 Issue 4336: HTTPRequest._send_output() now deals with the case of the message body not being a string. This allows clients to use endheaders(message_body) instead of endheaders() + send(message_body) without making any extra checks.
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 49890
diff changeset
802 #we must run the risk of Nagle
07bbc2e20aa4 Issue 4336: HTTPRequest._send_output() now deals with the case of the message body not being a string. This allows clients to use endheaders(message_body) instead of endheaders() + send(message_body) without making any extra checks.
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 49890
diff changeset
803 self.send(message_body)
24262
3b571e32a6e4 Send HTTP requests with a single send() call instead of many.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24216
diff changeset
804
30976
ead4c61219d0 Patch #831747: Add skip_accept_encoding parameter to putrequest.
Martin v. Löwis <martin@v.loewis.de>
parents: 30888
diff changeset
805 def putrequest(self, method, url, skip_host=0, skip_accept_encoding=0):
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
806 """Send a request to the server.
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
807
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
808 `method' specifies an HTTP request method, e.g. 'GET'.
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
809 `url' specifies the object being requested, e.g. '/index.html'.
30976
ead4c61219d0 Patch #831747: Add skip_accept_encoding parameter to putrequest.
Martin v. Löwis <martin@v.loewis.de>
parents: 30888
diff changeset
810 `skip_host' if True does not add automatically a 'Host:' header
ead4c61219d0 Patch #831747: Add skip_accept_encoding parameter to putrequest.
Martin v. Löwis <martin@v.loewis.de>
parents: 30888
diff changeset
811 `skip_accept_encoding' if True does not add automatically an
ead4c61219d0 Patch #831747: Add skip_accept_encoding parameter to putrequest.
Martin v. Löwis <martin@v.loewis.de>
parents: 30888
diff changeset
812 'Accept-Encoding:' header
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
813 """
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
814
29584
da0248fcfc52 Deal with a couple XXX comments which asked questions.
Greg Stein <gstein@lyra.org>
parents: 29465
diff changeset
815 # if a prior response has been completed, then forget about it.
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
816 if self.__response and self.__response.isclosed():
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
817 self.__response = None
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
818
31389
aca1ce1b4b61 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 30976
diff changeset
819
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
820 # in certain cases, we cannot issue another request on this connection.
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
821 # this occurs when:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
822 # 1) we are in the process of sending a request. (_CS_REQ_STARTED)
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
823 # 2) a response to a previous request has signalled that it is going
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
824 # to close the connection upon completion.
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
825 # 3) the headers for the previous response have not been read, thus
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
826 # we cannot determine whether point (2) is true. (_CS_REQ_SENT)
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
827 #
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
828 # if there is no prior response, then we can request at will.
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
829 #
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
830 # if point (2) is true, then we will have passed the socket to the
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
831 # response (effectively meaning, "there is no prior response"), and
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
832 # will open a new one when a new request is made.
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
833 #
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
834 # Note: if a prior response exists, then we *can* start a new request.
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
835 # We are not allowed to begin fetching the response to this new
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
836 # request, however, until that prior response is complete.
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
837 #
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
838 if self.__state == _CS_IDLE:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
839 self.__state = _CS_REQ_STARTED
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
840 else:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
841 raise CannotSendRequest()
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
842
28978
83fe0f5c9359 SF bug 622042: Don't expect response body from HEAD request.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 28671
diff changeset
843 # Save the method we use, we need it later in the response phase
83fe0f5c9359 SF bug 622042: Don't expect response body from HEAD request.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 28671
diff changeset
844 self._method = method
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
845 if not url:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
846 url = '/'
24262
3b571e32a6e4 Send HTTP requests with a single send() call instead of many.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24216
diff changeset
847 str = '%s %s %s' % (method, url, self._http_vsn_str)
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
848
24262
3b571e32a6e4 Send HTTP requests with a single send() call instead of many.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24216
diff changeset
849 self._output(str)
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
850
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
851 if self._http_vsn == 11:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
852 # Issue some standard headers for better HTTP/1.1 compliance
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
853
22202
5f2c0f6f0af7 Fix SF bug 525520.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 22192
diff changeset
854 if not skip_host:
5f2c0f6f0af7 Fix SF bug 525520.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 22192
diff changeset
855 # this header is issued *only* for HTTP/1.1
5f2c0f6f0af7 Fix SF bug 525520.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 22192
diff changeset
856 # connections. more specifically, this means it is
5f2c0f6f0af7 Fix SF bug 525520.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 22192
diff changeset
857 # only issued when the client uses the new
5f2c0f6f0af7 Fix SF bug 525520.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 22192
diff changeset
858 # HTTPConnection() class. backwards-compat clients
5f2c0f6f0af7 Fix SF bug 525520.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 22192
diff changeset
859 # will be using HTTP/1.0 and those clients may be
5f2c0f6f0af7 Fix SF bug 525520.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 22192
diff changeset
860 # issuing this header themselves. we should NOT issue
5f2c0f6f0af7 Fix SF bug 525520.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 22192
diff changeset
861 # it twice; some web servers (such as Apache) barf
5f2c0f6f0af7 Fix SF bug 525520.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 22192
diff changeset
862 # when they see two Host: headers
15859
31967a233600 SF Patch #103232 by dougfort: Preserve Nonstandard Port Number in Host
Guido van Rossum <guido@python.org>
parents: 15845
diff changeset
863
22202
5f2c0f6f0af7 Fix SF bug 525520.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 22192
diff changeset
864 # If we need a non-standard port,include it in the
5f2c0f6f0af7 Fix SF bug 525520.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 22192
diff changeset
865 # header. If the request is going through a proxy,
5f2c0f6f0af7 Fix SF bug 525520.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 22192
diff changeset
866 # but the host of the actual URL, not the host of the
5f2c0f6f0af7 Fix SF bug 525520.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 22192
diff changeset
867 # proxy.
22192
044f9e71320f SF bug report #405939: wrong Host header with proxy
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 21962
diff changeset
868
22202
5f2c0f6f0af7 Fix SF bug 525520.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 22192
diff changeset
869 netloc = ''
5f2c0f6f0af7 Fix SF bug 525520.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 22192
diff changeset
870 if url.startswith('http'):
5f2c0f6f0af7 Fix SF bug 525520.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 22192
diff changeset
871 nil, netloc, nil, nil, nil = urlsplit(url)
5f2c0f6f0af7 Fix SF bug 525520.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 22192
diff changeset
872
5f2c0f6f0af7 Fix SF bug 525520.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 22192
diff changeset
873 if netloc:
38104
88b89f96b97f RFE #1472176: In httplib, don't encode the netloc and hostname with "idna" if not necessary.
Georg Brandl <georg@python.org>
parents: 36559
diff changeset
874 try:
88b89f96b97f RFE #1472176: In httplib, don't encode the netloc and hostname with "idna" if not necessary.
Georg Brandl <georg@python.org>
parents: 36559
diff changeset
875 netloc_enc = netloc.encode("ascii")
88b89f96b97f RFE #1472176: In httplib, don't encode the netloc and hostname with "idna" if not necessary.
Georg Brandl <georg@python.org>
parents: 36559
diff changeset
876 except UnicodeEncodeError:
88b89f96b97f RFE #1472176: In httplib, don't encode the netloc and hostname with "idna" if not necessary.
Georg Brandl <georg@python.org>
parents: 36559
diff changeset
877 netloc_enc = netloc.encode("idna")
88b89f96b97f RFE #1472176: In httplib, don't encode the netloc and hostname with "idna" if not necessary.
Georg Brandl <georg@python.org>
parents: 36559
diff changeset
878 self.putheader('Host', netloc_enc)
22202
5f2c0f6f0af7 Fix SF bug 525520.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 22192
diff changeset
879 else:
38104
88b89f96b97f RFE #1472176: In httplib, don't encode the netloc and hostname with "idna" if not necessary.
Georg Brandl <georg@python.org>
parents: 36559
diff changeset
880 try:
88b89f96b97f RFE #1472176: In httplib, don't encode the netloc and hostname with "idna" if not necessary.
Georg Brandl <georg@python.org>
parents: 36559
diff changeset
881 host_enc = self.host.encode("ascii")
88b89f96b97f RFE #1472176: In httplib, don't encode the netloc and hostname with "idna" if not necessary.
Georg Brandl <georg@python.org>
parents: 36559
diff changeset
882 except UnicodeEncodeError:
88b89f96b97f RFE #1472176: In httplib, don't encode the netloc and hostname with "idna" if not necessary.
Georg Brandl <georg@python.org>
parents: 36559
diff changeset
883 host_enc = self.host.encode("idna")
47909
08c5c8c114b2 - Issue #3094: httplib.HTTPSConnection Host: headers no longer include the
Gregory P. Smith <greg@mad-scientist.com>
parents: 47372
diff changeset
884 if self.port == self.default_port:
38104
88b89f96b97f RFE #1472176: In httplib, don't encode the netloc and hostname with "idna" if not necessary.
Georg Brandl <georg@python.org>
parents: 36559
diff changeset
885 self.putheader('Host', host_enc)
88b89f96b97f RFE #1472176: In httplib, don't encode the netloc and hostname with "idna" if not necessary.
Georg Brandl <georg@python.org>
parents: 36559
diff changeset
886 else:
88b89f96b97f RFE #1472176: In httplib, don't encode the netloc and hostname with "idna" if not necessary.
Georg Brandl <georg@python.org>
parents: 36559
diff changeset
887 self.putheader('Host', "%s:%s" % (host_enc, self.port))
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
888
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
889 # note: we are assuming that clients will not attempt to set these
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
890 # headers since *this* library must deal with the
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
891 # consequences. this also means that when the supporting
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
892 # libraries are updated to recognize other forms, then this
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
893 # code should be changed (removed or updated).
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
894
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
895 # we only want a Content-Encoding of "identity" since we don't
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
896 # support encodings such as x-gzip or x-deflate.
30976
ead4c61219d0 Patch #831747: Add skip_accept_encoding parameter to putrequest.
Martin v. Löwis <martin@v.loewis.de>
parents: 30888
diff changeset
897 if not skip_accept_encoding:
ead4c61219d0 Patch #831747: Add skip_accept_encoding parameter to putrequest.
Martin v. Löwis <martin@v.loewis.de>
parents: 30888
diff changeset
898 self.putheader('Accept-Encoding', 'identity')
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
899
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
900 # we can accept "chunked" Transfer-Encodings, but no others
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
901 # NOTE: no TE header implies *only* "chunked"
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
902 #self.putheader('TE', 'chunked')
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
903
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
904 # if TE is supplied in the header, then it must appear in a
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
905 # Connection header.
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
906 #self.putheader('Connection', 'TE')
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
907
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
908 else:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
909 # For HTTP/1.0, the server will assume "not chunked"
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
910 pass
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
911
49726
0ad07078433a The docs for httplib.HTTPConnection.putheader() have claimed for quite a while
Brett Cannon <bcannon@gmail.com>
parents: 48948
diff changeset
912 def putheader(self, header, *values):
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
913 """Send a request header line to the server.
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
914
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
915 For example: h.putheader('Accept', 'text/html')
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
916 """
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
917 if self.__state != _CS_REQ_STARTED:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
918 raise CannotSendHeader()
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
919
49726
0ad07078433a The docs for httplib.HTTPConnection.putheader() have claimed for quite a while
Brett Cannon <bcannon@gmail.com>
parents: 48948
diff changeset
920 str = '%s: %s' % (header, '\r\n\t'.join(values))
24262
3b571e32a6e4 Send HTTP requests with a single send() call instead of many.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24216
diff changeset
921 self._output(str)
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
922
49890
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
923 def endheaders(self, message_body=None):
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
924 """Indicate that the last header line has been sent to the server.
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
925
49890
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
926 This method sends the request to the server. The optional
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
927 message_body argument can be used to pass message body
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
928 associated with the request. The message body will be sent in
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
929 the same packet as the message headers if possible. The
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
930 message_body should be a string.
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
931 """
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
932 if self.__state == _CS_REQ_STARTED:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
933 self.__state = _CS_REQ_SENT
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
934 else:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
935 raise CannotSendHeader()
49890
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
936 self._send_output(message_body)
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
937
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
938 def request(self, method, url, body=None, headers={}):
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
939 """Send a complete request to the server."""
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
940
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
941 try:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
942 self._send_request(method, url, body, headers)
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
943 except socket.error, v:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
944 # trap 'Broken pipe' if we're allowed to automatically reconnect
56528
64e19a689886 Fix py3k warnings in httplib
Antoine Pitrou <solipsis@pitrou.net>
parents: 56285
diff changeset
945 if v.args[0] != 32 or not self.auto_open:
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
946 raise
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
947 # try one more time
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
948 self._send_request(method, url, body, headers)
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
949
49890
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
950 def _set_content_length(self, body):
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
951 # Set the content-length based on the body.
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
952 thelen = None
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
953 try:
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
954 thelen = str(len(body))
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
955 except TypeError, te:
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
956 # If this is a file-like object, try to
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
957 # fstat its file descriptor
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
958 import os
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
959 try:
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
960 thelen = str(os.fstat(body.fileno()).st_size)
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
961 except (AttributeError, OSError):
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
962 # Don't send a length if this failed
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
963 if self.debuglevel > 0: print "Cannot stat!!"
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
964
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
965 if thelen is not None:
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
966 self.putheader('Content-Length', thelen)
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
967
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
968 def _send_request(self, method, url, body, headers):
33001
4fd59651fe3f SF bug 874842 and patch 997626: httplib bugs
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 32252
diff changeset
969 # honour explicitly requested Host: and Accept-Encoding headers
4fd59651fe3f SF bug 874842 and patch 997626: httplib bugs
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 32252
diff changeset
970 header_names = dict.fromkeys([k.lower() for k in headers])
4fd59651fe3f SF bug 874842 and patch 997626: httplib bugs
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 32252
diff changeset
971 skips = {}
4fd59651fe3f SF bug 874842 and patch 997626: httplib bugs
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 32252
diff changeset
972 if 'host' in header_names:
4fd59651fe3f SF bug 874842 and patch 997626: httplib bugs
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 32252
diff changeset
973 skips['skip_host'] = 1
4fd59651fe3f SF bug 874842 and patch 997626: httplib bugs
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 32252
diff changeset
974 if 'accept-encoding' in header_names:
4fd59651fe3f SF bug 874842 and patch 997626: httplib bugs
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 32252
diff changeset
975 skips['skip_accept_encoding'] = 1
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
976
33001
4fd59651fe3f SF bug 874842 and patch 997626: httplib bugs
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 32252
diff changeset
977 self.putrequest(method, url, **skips)
4fd59651fe3f SF bug 874842 and patch 997626: httplib bugs
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 32252
diff changeset
978
4fd59651fe3f SF bug 874842 and patch 997626: httplib bugs
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 32252
diff changeset
979 if body and ('content-length' not in header_names):
49890
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
980 self._set_content_length(body)
28083
29965d0d4650 Module review:
Raymond Hettinger <python@rcn.com>
parents: 28063
diff changeset
981 for hdr, value in headers.iteritems():
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
982 self.putheader(hdr, value)
50672
07bbc2e20aa4 Issue 4336: HTTPRequest._send_output() now deals with the case of the message body not being a string. This allows clients to use endheaders(message_body) instead of endheaders() + send(message_body) without making any extra checks.
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 49890
diff changeset
983 self.endheaders(body)
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
984
50734
d09d6fe31b61 Issue 4879: Allow buffering for HTTPResponse
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 50672
diff changeset
985 def getresponse(self, buffering=False):
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
986 "Get the response from the server."
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
987
29584
da0248fcfc52 Deal with a couple XXX comments which asked questions.
Greg Stein <gstein@lyra.org>
parents: 29465
diff changeset
988 # if a prior response has been completed, then forget about it.
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
989 if self.__response and self.__response.isclosed():
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
990 self.__response = None
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
991
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
992 #
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
993 # if a prior response exists, then it must be completed (otherwise, we
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
994 # cannot read this response's header to determine the connection-close
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
995 # behavior)
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
996 #
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
997 # note: if a prior response existed, but was connection-close, then the
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
998 # socket and response were made independent of this HTTPConnection
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
999 # object since a new request requires that we open a whole new
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1000 # connection
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1001 #
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1002 # this means the prior response had one of two states:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1003 # 1) will_close: this connection was reset and the prior socket and
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1004 # response operate independently
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1005 # 2) persistent: the response was retained and we await its
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1006 # isclosed() status to become true.
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1007 #
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1008 if self.__state != _CS_REQ_SENT or self.__response:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1009 raise ResponseNotReady()
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1010
50734
d09d6fe31b61 Issue 4879: Allow buffering for HTTPResponse
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 50672
diff changeset
1011 args = (self.sock,)
d09d6fe31b61 Issue 4879: Allow buffering for HTTPResponse
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 50672
diff changeset
1012 kwds = {"strict":self.strict, "method":self._method}
14672
1b20ad6eb501 Do not close socket when a Content-Length is 0. This make the
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14560
diff changeset
1013 if self.debuglevel > 0:
50734
d09d6fe31b61 Issue 4879: Allow buffering for HTTPResponse
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 50672
diff changeset
1014 args += (self.debuglevel,)
d09d6fe31b61 Issue 4879: Allow buffering for HTTPResponse
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 50672
diff changeset
1015 if buffering:
d09d6fe31b61 Issue 4879: Allow buffering for HTTPResponse
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 50672
diff changeset
1016 #only add this keyword if non-default, for compatibility with
d09d6fe31b61 Issue 4879: Allow buffering for HTTPResponse
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 50672
diff changeset
1017 #other response_classes.
d09d6fe31b61 Issue 4879: Allow buffering for HTTPResponse
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 50672
diff changeset
1018 kwds["buffering"] = True;
d09d6fe31b61 Issue 4879: Allow buffering for HTTPResponse
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 50672
diff changeset
1019 response = self.response_class(*args, **kwds)
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1020
24216
c8a19a428358 Change _begin() back to begin().
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24168
diff changeset
1021 response.begin()
24038
9377158652dd Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23620
diff changeset
1022 assert response.will_close != _UNKNOWN
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1023 self.__state = _CS_IDLE
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1024
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1025 if response.will_close:
41285
02574f485b15 Bug #978833: Revert r50844, as it broke _socketobject.dup.
Martin v. Löwis <martin@v.loewis.de>
parents: 40732
diff changeset
1026 # this effectively passes the connection to the response
02574f485b15 Bug #978833: Revert r50844, as it broke _socketobject.dup.
Martin v. Löwis <martin@v.loewis.de>
parents: 40732
diff changeset
1027 self.close()
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1028 else:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1029 # remember this, so we can tell when it is complete
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1030 self.__response = response
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1031
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1032 return response
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1033
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1034
13787
1f966f0d467c add support for HTTPS
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 13636
diff changeset
1035 class HTTP:
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1036 "Compatibility class with httplib.py from 1.5."
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1037
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1038 _http_vsn = 10
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1039 _http_vsn_str = 'HTTP/1.0'
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1040
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1041 debuglevel = 0
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1042
13787
1f966f0d467c add support for HTTPS
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 13636
diff changeset
1043 _connection_class = HTTPConnection
1f966f0d467c add support for HTTPS
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 13636
diff changeset
1044
24119
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1045 def __init__(self, host='', port=None, strict=None):
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1046 "Provide a default host, since the superclass requires one."
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1047
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1048 # some joker passed 0 explicitly, meaning default port
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1049 if port == 0:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1050 port = None
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1051
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1052 # Note that we may pass an empty string as the host; this will throw
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1053 # an error when we attempt to connect. Presumably, the client code
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1054 # will call connect before then, with a proper host.
24119
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1055 self._setup(self._connection_class(host, port, strict))
19367
17802077e77c Resolve patch #449367.
Greg Stein <gstein@lyra.org>
parents: 18902
diff changeset
1056
17802077e77c Resolve patch #449367.
Greg Stein <gstein@lyra.org>
parents: 18902
diff changeset
1057 def _setup(self, conn):
17802077e77c Resolve patch #449367.
Greg Stein <gstein@lyra.org>
parents: 18902
diff changeset
1058 self._conn = conn
17802077e77c Resolve patch #449367.
Greg Stein <gstein@lyra.org>
parents: 18902
diff changeset
1059
13787
1f966f0d467c add support for HTTPS
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 13636
diff changeset
1060 # set up delegation to flesh out interface
19367
17802077e77c Resolve patch #449367.
Greg Stein <gstein@lyra.org>
parents: 18902
diff changeset
1061 self.send = conn.send
17802077e77c Resolve patch #449367.
Greg Stein <gstein@lyra.org>
parents: 18902
diff changeset
1062 self.putrequest = conn.putrequest
49726
0ad07078433a The docs for httplib.HTTPConnection.putheader() have claimed for quite a while
Brett Cannon <bcannon@gmail.com>
parents: 48948
diff changeset
1063 self.putheader = conn.putheader
19367
17802077e77c Resolve patch #449367.
Greg Stein <gstein@lyra.org>
parents: 18902
diff changeset
1064 self.endheaders = conn.endheaders
17802077e77c Resolve patch #449367.
Greg Stein <gstein@lyra.org>
parents: 18902
diff changeset
1065 self.set_debuglevel = conn.set_debuglevel
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1066
19367
17802077e77c Resolve patch #449367.
Greg Stein <gstein@lyra.org>
parents: 18902
diff changeset
1067 conn._http_vsn = self._http_vsn
17802077e77c Resolve patch #449367.
Greg Stein <gstein@lyra.org>
parents: 18902
diff changeset
1068 conn._http_vsn_str = self._http_vsn_str
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1069
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1070 self.file = None
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1071
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1072 def connect(self, host=None, port=None):
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1073 "Accept arguments to set the host/port, since the superclass doesn't."
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1074
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1075 if host is not None:
13787
1f966f0d467c add support for HTTPS
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 13636
diff changeset
1076 self._conn._set_hostport(host, port)
1f966f0d467c add support for HTTPS
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 13636
diff changeset
1077 self._conn.connect()
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1078
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1079 def getfile(self):
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1080 "Provide a getfile, since the superclass' does not use this concept."
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1081 return self.file
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1082
50734
d09d6fe31b61 Issue 4879: Allow buffering for HTTPResponse
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 50672
diff changeset
1083 def getreply(self, buffering=False):
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1084 """Compat definition since superclass does not define it.
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1085
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1086 Returns a tuple consisting of:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1087 - server status code (e.g. '200' if all goes well)
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1088 - server "reason" corresponding to status code
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1089 - any RFC822 headers in the response from the server
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1090 """
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1091 try:
50734
d09d6fe31b61 Issue 4879: Allow buffering for HTTPResponse
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 50672
diff changeset
1092 if not buffering:
d09d6fe31b61 Issue 4879: Allow buffering for HTTPResponse
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 50672
diff changeset
1093 response = self._conn.getresponse()
d09d6fe31b61 Issue 4879: Allow buffering for HTTPResponse
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 50672
diff changeset
1094 else:
d09d6fe31b61 Issue 4879: Allow buffering for HTTPResponse
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 50672
diff changeset
1095 #only add this keyword if non-default for compatibility
d09d6fe31b61 Issue 4879: Allow buffering for HTTPResponse
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 50672
diff changeset
1096 #with other connection classes
d09d6fe31b61 Issue 4879: Allow buffering for HTTPResponse
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 50672
diff changeset
1097 response = self._conn.getresponse(buffering)
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1098 except BadStatusLine, e:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1099 ### hmm. if getresponse() ever closes the socket on a bad request,
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1100 ### then we are going to have problems with self.sock
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1101
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1102 ### should we keep this behavior? do people use it?
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1103 # keep the socket open (as a file), and return it
13787
1f966f0d467c add support for HTTPS
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 13636
diff changeset
1104 self.file = self._conn.sock.makefile('rb', 0)
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1105
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1106 # close our socket -- we want to restart after any protocol error
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1107 self.close()
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1108
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1109 self.headers = None
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1110 return -1, e.line, None
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1111
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1112 self.headers = response.msg
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1113 self.file = response.fp
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1114 return response.status, response.reason, response.msg
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1115
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1116 def close(self):
13787
1f966f0d467c add support for HTTPS
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 13636
diff changeset
1117 self._conn.close()
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1118
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1119 # note that self.file == response.fp, which gets closed by the
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1120 # superclass. just clear the object ref here.
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1121 ### hmm. messy. if status==-1, then self.file is owned by us.
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1122 ### well... we aren't explicitly closing, but losing this ref will
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1123 ### do it
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1124 self.file = None
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1125
43008
4771639fc049 This contains a number of things:
Bill Janssen <janssen@parc.com>
parents: 42385
diff changeset
1126 try:
4771639fc049 This contains a number of things:
Bill Janssen <janssen@parc.com>
parents: 42385
diff changeset
1127 import ssl
4771639fc049 This contains a number of things:
Bill Janssen <janssen@parc.com>
parents: 42385
diff changeset
1128 except ImportError:
4771639fc049 This contains a number of things:
Bill Janssen <janssen@parc.com>
parents: 42385
diff changeset
1129 pass
4771639fc049 This contains a number of things:
Bill Janssen <janssen@parc.com>
parents: 42385
diff changeset
1130 else:
4771639fc049 This contains a number of things:
Bill Janssen <janssen@parc.com>
parents: 42385
diff changeset
1131 class HTTPSConnection(HTTPConnection):
4771639fc049 This contains a number of things:
Bill Janssen <janssen@parc.com>
parents: 42385
diff changeset
1132 "This class allows communication via SSL."
4771639fc049 This contains a number of things:
Bill Janssen <janssen@parc.com>
parents: 42385
diff changeset
1133
4771639fc049 This contains a number of things:
Bill Janssen <janssen@parc.com>
parents: 42385
diff changeset
1134 default_port = HTTPS_PORT
4771639fc049 This contains a number of things:
Bill Janssen <janssen@parc.com>
parents: 42385
diff changeset
1135
4771639fc049 This contains a number of things:
Bill Janssen <janssen@parc.com>
parents: 42385
diff changeset
1136 def __init__(self, host, port=None, key_file=None, cert_file=None,
58113
a19ad299095f issue3972: HTTPConnection and HTTPSConnection now support a
Gregory P. Smith <greg@mad-scientist.com>
parents: 57799
diff changeset
1137 strict=None, timeout=socket._GLOBAL_DEFAULT_TIMEOUT,
a19ad299095f issue3972: HTTPConnection and HTTPSConnection now support a
Gregory P. Smith <greg@mad-scientist.com>
parents: 57799
diff changeset
1138 source_address=None):
a19ad299095f issue3972: HTTPConnection and HTTPSConnection now support a
Gregory P. Smith <greg@mad-scientist.com>
parents: 57799
diff changeset
1139 HTTPConnection.__init__(self, host, port, strict, timeout,
a19ad299095f issue3972: HTTPConnection and HTTPSConnection now support a
Gregory P. Smith <greg@mad-scientist.com>
parents: 57799
diff changeset
1140 source_address)
43008
4771639fc049 This contains a number of things:
Bill Janssen <janssen@parc.com>
parents: 42385
diff changeset
1141 self.key_file = key_file
4771639fc049 This contains a number of things:
Bill Janssen <janssen@parc.com>
parents: 42385
diff changeset
1142 self.cert_file = cert_file
4771639fc049 This contains a number of things:
Bill Janssen <janssen@parc.com>
parents: 42385
diff changeset
1143
4771639fc049 This contains a number of things:
Bill Janssen <janssen@parc.com>
parents: 42385
diff changeset
1144 def connect(self):
4771639fc049 This contains a number of things:
Bill Janssen <janssen@parc.com>
parents: 42385
diff changeset
1145 "Connect to a host on a given (SSL) port."
4771639fc049 This contains a number of things:
Bill Janssen <janssen@parc.com>
parents: 42385
diff changeset
1146
58113
a19ad299095f issue3972: HTTPConnection and HTTPSConnection now support a
Gregory P. Smith <greg@mad-scientist.com>
parents: 57799
diff changeset
1147 sock = socket.create_connection((self.host, self.port),
a19ad299095f issue3972: HTTPConnection and HTTPSConnection now support a
Gregory P. Smith <greg@mad-scientist.com>
parents: 57799
diff changeset
1148 self.timeout, self.source_address)
54298
e7b4a443f764 Fixed Issue1424152, urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 51951
diff changeset
1149 if self._tunnel_host:
e7b4a443f764 Fixed Issue1424152, urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 51951
diff changeset
1150 self.sock = sock
e7b4a443f764 Fixed Issue1424152, urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 51951
diff changeset
1151 self._tunnel()
43325
193020bf52dc More work on SSL support.
Bill Janssen <janssen@parc.com>
parents: 43161
diff changeset
1152 self.sock = ssl.wrap_socket(sock, self.key_file, self.cert_file)
43008
4771639fc049 This contains a number of things:
Bill Janssen <janssen@parc.com>
parents: 42385
diff changeset
1153
43093
1d57fb7efb15 Don't lie in __all__ attributes when SSL is not available: only add the SSL
Thomas Wouters <thomas@python.org>
parents: 43008
diff changeset
1154 __all__.append("HTTPSConnection")
43008
4771639fc049 This contains a number of things:
Bill Janssen <janssen@parc.com>
parents: 42385
diff changeset
1155
13787
1f966f0d467c add support for HTTPS
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 13636
diff changeset
1156 class HTTPS(HTTP):
1f966f0d467c add support for HTTPS
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 13636
diff changeset
1157 """Compatibility with 1.5 httplib interface
1f966f0d467c add support for HTTPS
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 13636
diff changeset
1158
1f966f0d467c add support for HTTPS
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 13636
diff changeset
1159 Python 1.5.2 did not have an HTTPS class, but it defined an
1f966f0d467c add support for HTTPS
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 13636
diff changeset
1160 interface for sending http requests that is also useful for
15845
210bbe53b6d4 No text file relying on significant trailing whitespace is robust under
Tim Peters <tim.peters@gmail.com>
parents: 15844
diff changeset
1161 https.
13787
1f966f0d467c add support for HTTPS
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 13636
diff changeset
1162 """
1f966f0d467c add support for HTTPS
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 13636
diff changeset
1163
14740
8bb6dca66ecd Indent _connection_class so that it becomes HTTPS._connection_class.
Martin v. Löwis <martin@v.loewis.de>
parents: 14672
diff changeset
1164 _connection_class = HTTPSConnection
15845
210bbe53b6d4 No text file relying on significant trailing whitespace is robust under
Tim Peters <tim.peters@gmail.com>
parents: 15844
diff changeset
1165
24119
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1166 def __init__(self, host='', port=None, key_file=None, cert_file=None,
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1167 strict=None):
19367
17802077e77c Resolve patch #449367.
Greg Stein <gstein@lyra.org>
parents: 18902
diff changeset
1168 # provide a default host, pass the X509 cert info
17802077e77c Resolve patch #449367.
Greg Stein <gstein@lyra.org>
parents: 18902
diff changeset
1169
17802077e77c Resolve patch #449367.
Greg Stein <gstein@lyra.org>
parents: 18902
diff changeset
1170 # urf. compensate for bad input.
17802077e77c Resolve patch #449367.
Greg Stein <gstein@lyra.org>
parents: 18902
diff changeset
1171 if port == 0:
17802077e77c Resolve patch #449367.
Greg Stein <gstein@lyra.org>
parents: 18902
diff changeset
1172 port = None
24119
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1173 self._setup(self._connection_class(host, port, key_file,
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1174 cert_file, strict))
19367
17802077e77c Resolve patch #449367.
Greg Stein <gstein@lyra.org>
parents: 18902
diff changeset
1175
17802077e77c Resolve patch #449367.
Greg Stein <gstein@lyra.org>
parents: 18902
diff changeset
1176 # we never actually use these for anything, but we keep them
17802077e77c Resolve patch #449367.
Greg Stein <gstein@lyra.org>
parents: 18902
diff changeset
1177 # here for compatibility with post-1.5.2 CVS.
24119
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1178 self.key_file = key_file
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1179 self.cert_file = cert_file
19367
17802077e77c Resolve patch #449367.
Greg Stein <gstein@lyra.org>
parents: 18902
diff changeset
1180
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1181
43008
4771639fc049 This contains a number of things:
Bill Janssen <janssen@parc.com>
parents: 42385
diff changeset
1182 def FakeSocket (sock, sslobj):
43161
9cfad9a5c83b deprecate use of FakeSocket
Bill Janssen <janssen@parc.com>
parents: 43093
diff changeset
1183 warnings.warn("FakeSocket is deprecated, and won't be in 3.x. " +
43325
193020bf52dc More work on SSL support.
Bill Janssen <janssen@parc.com>
parents: 43161
diff changeset
1184 "Use the result of ssl.wrap_socket() directly instead.",
43161
9cfad9a5c83b deprecate use of FakeSocket
Bill Janssen <janssen@parc.com>
parents: 43093
diff changeset
1185 DeprecationWarning, stacklevel=2)
43008
4771639fc049 This contains a number of things:
Bill Janssen <janssen@parc.com>
parents: 42385
diff changeset
1186 return sslobj
4771639fc049 This contains a number of things:
Bill Janssen <janssen@parc.com>
parents: 42385
diff changeset
1187
4771639fc049 This contains a number of things:
Bill Janssen <janssen@parc.com>
parents: 42385
diff changeset
1188
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1189 class HTTPException(Exception):
24120
b51a578f5769 Fix SF bug #575360
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24119
diff changeset
1190 # Subclasses that define an __init__ must call Exception.__init__
b51a578f5769 Fix SF bug #575360
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24119
diff changeset
1191 # or define self.args. Otherwise, str() will fail.
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1192 pass
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1193
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1194 class NotConnected(HTTPException):
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1195 pass
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1196
22494
7224a1df4809 add InvalidURL exception - raised if port is given but empty or non-numeric
Skip Montanaro <skip@pobox.com>
parents: 22403
diff changeset
1197 class InvalidURL(HTTPException):
7224a1df4809 add InvalidURL exception - raised if port is given but empty or non-numeric
Skip Montanaro <skip@pobox.com>
parents: 22403
diff changeset
1198 pass
7224a1df4809 add InvalidURL exception - raised if port is given but empty or non-numeric
Skip Montanaro <skip@pobox.com>
parents: 22403
diff changeset
1199
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1200 class UnknownProtocol(HTTPException):
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1201 def __init__(self, version):
24120
b51a578f5769 Fix SF bug #575360
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24119
diff changeset
1202 self.args = version,
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1203 self.version = version
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1204
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1205 class UnknownTransferEncoding(HTTPException):
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1206 pass
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1207
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1208 class UnimplementedFileMode(HTTPException):
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1209 pass
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1210
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1211 class IncompleteRead(HTTPException):
51951
87115c727102 give httplib.IncompleteRead a more sane repr #4308
Benjamin Peterson <benjamin@python.org>
parents: 50734
diff changeset
1212 def __init__(self, partial, expected=None):
24120
b51a578f5769 Fix SF bug #575360
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24119
diff changeset
1213 self.args = partial,
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1214 self.partial = partial
51951
87115c727102 give httplib.IncompleteRead a more sane repr #4308
Benjamin Peterson <benjamin@python.org>
parents: 50734
diff changeset
1215 self.expected = expected
87115c727102 give httplib.IncompleteRead a more sane repr #4308
Benjamin Peterson <benjamin@python.org>
parents: 50734
diff changeset
1216 def __repr__(self):
87115c727102 give httplib.IncompleteRead a more sane repr #4308
Benjamin Peterson <benjamin@python.org>
parents: 50734
diff changeset
1217 if self.expected is not None:
87115c727102 give httplib.IncompleteRead a more sane repr #4308
Benjamin Peterson <benjamin@python.org>
parents: 50734
diff changeset
1218 e = ', %i more expected' % self.expected
87115c727102 give httplib.IncompleteRead a more sane repr #4308
Benjamin Peterson <benjamin@python.org>
parents: 50734
diff changeset
1219 else:
87115c727102 give httplib.IncompleteRead a more sane repr #4308
Benjamin Peterson <benjamin@python.org>
parents: 50734
diff changeset
1220 e = ''
87115c727102 give httplib.IncompleteRead a more sane repr #4308
Benjamin Peterson <benjamin@python.org>
parents: 50734
diff changeset
1221 return 'IncompleteRead(%i bytes read%s)' % (len(self.partial), e)
87115c727102 give httplib.IncompleteRead a more sane repr #4308
Benjamin Peterson <benjamin@python.org>
parents: 50734
diff changeset
1222 def __str__(self):
87115c727102 give httplib.IncompleteRead a more sane repr #4308
Benjamin Peterson <benjamin@python.org>
parents: 50734
diff changeset
1223 return repr(self)
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1224
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1225 class ImproperConnectionState(HTTPException):
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1226 pass
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1227
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1228 class CannotSendRequest(ImproperConnectionState):
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1229 pass
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1230
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1231 class CannotSendHeader(ImproperConnectionState):
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1232 pass
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1233
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1234 class ResponseNotReady(ImproperConnectionState):
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1235 pass
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1236
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1237 class BadStatusLine(HTTPException):
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1238 def __init__(self, line):
24120
b51a578f5769 Fix SF bug #575360
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24119
diff changeset
1239 self.args = line,
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1240 self.line = line
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1241
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1242 # for backwards compatibility
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1243 error = HTTPException
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1244
24119
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1245 class LineAndFileWrapper:
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1246 """A limited file-like object for HTTP/0.9 responses."""
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1247
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1248 # The status-line parsing code calls readline(), which normally
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1249 # get the HTTP status line. For a 0.9 response, however, this is
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1250 # actually the first line of the body! Clients need to get a
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1251 # readable file object that contains that line.
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1252
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1253 def __init__(self, line, file):
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1254 self._line = line
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1255 self._file = file
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1256 self._line_consumed = 0
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1257 self._line_offset = 0
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1258 self._line_left = len(line)
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1259
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1260 def __getattr__(self, attr):
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1261 return getattr(self._file, attr)
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1262
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1263 def _done(self):
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1264 # called when the last byte is read from the line. After the
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1265 # call, all read methods are delegated to the underlying file
28063
Skip Montanaro <skip@pobox.com>
parents: 26387
diff changeset
1266 # object.
24119
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1267 self._line_consumed = 1
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1268 self.read = self._file.read
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1269 self.readline = self._file.readline
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1270 self.readlines = self._file.readlines
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1271
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1272 def read(self, amt=None):
32252
a72bc4257336 Fix a bug that robotparser starves memory when the server responses
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31967
diff changeset
1273 if self._line_consumed:
a72bc4257336 Fix a bug that robotparser starves memory when the server responses
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31967
diff changeset
1274 return self._file.read(amt)
a72bc4257336 Fix a bug that robotparser starves memory when the server responses
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31967
diff changeset
1275 assert self._line_left
24119
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1276 if amt is None or amt > self._line_left:
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1277 s = self._line[self._line_offset:]
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1278 self._done()
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1279 if amt is None:
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1280 return s + self._file.read()
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1281 else:
24263
e6033254df21 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 24262
diff changeset
1282 return s + self._file.read(amt - len(s))
24119
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1283 else:
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1284 assert amt <= self._line_left
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1285 i = self._line_offset
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1286 j = i + amt
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1287 s = self._line[i:j]
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1288 self._line_offset = j
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1289 self._line_left -= amt
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1290 if self._line_left == 0:
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1291 self._done()
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1292 return s
24263
e6033254df21 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 24262
diff changeset
1293
24119
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1294 def readline(self):
32252
a72bc4257336 Fix a bug that robotparser starves memory when the server responses
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31967
diff changeset
1295 if self._line_consumed:
a72bc4257336 Fix a bug that robotparser starves memory when the server responses
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31967
diff changeset
1296 return self._file.readline()
a72bc4257336 Fix a bug that robotparser starves memory when the server responses
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31967
diff changeset
1297 assert self._line_left
24119
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1298 s = self._line[self._line_offset:]
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1299 self._done()
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1300 return s
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1301
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1302 def readlines(self, size=None):
32252
a72bc4257336 Fix a bug that robotparser starves memory when the server responses
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31967
diff changeset
1303 if self._line_consumed:
a72bc4257336 Fix a bug that robotparser starves memory when the server responses
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31967
diff changeset
1304 return self._file.readlines(size)
a72bc4257336 Fix a bug that robotparser starves memory when the server responses
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31967
diff changeset
1305 assert self._line_left
24119
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1306 L = [self._line[self._line_offset:]]
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1307 self._done()
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1308 if size is None:
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1309 return L + self._file.readlines()
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1310 else:
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1311 return L + self._file.readlines(size)
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1312
1555
6d35308261f4 Towards a standard access mechanism for URLs.
Guido van Rossum <guido@python.org>
parents:
diff changeset
1313 def test():
6787
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
1314 """Test this module.
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
1315
24168
fdd836575248 Fix for SF bug 579107.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24123
diff changeset
1316 A hodge podge of tests collected here, because they have too many
fdd836575248 Fix for SF bug 579107.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24123
diff changeset
1317 external dependencies for the regular test suite.
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1318 """
6787
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
1319
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
1320 import sys
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
1321 import getopt
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
1322 opts, args = getopt.getopt(sys.argv[1:], 'd')
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
1323 dl = 0
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
1324 for o, a in opts:
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
1325 if o == '-d': dl = dl + 1
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
1326 host = 'www.python.org'
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
1327 selector = '/'
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
1328 if args[0:]: host = args[0]
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
1329 if args[1:]: selector = args[1]
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
1330 h = HTTP()
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
1331 h.set_debuglevel(dl)
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
1332 h.connect(host)
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
1333 h.putrequest('GET', selector)
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
1334 h.endheaders()
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1335 status, reason, headers = h.getreply()
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1336 print 'status =', status
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1337 print 'reason =', reason
24168
fdd836575248 Fix for SF bug 579107.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24123
diff changeset
1338 print "read", len(h.getfile().read())
6787
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
1339 print
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
1340 if headers:
15582
6179d4afa4e4 Get rid of string functions.
Guido van Rossum <guido@python.org>
parents: 15503
diff changeset
1341 for header in headers.headers: print header.strip()
6787
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
1342 print
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1343
22192
044f9e71320f SF bug report #405939: wrong Host header with proxy
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 21962
diff changeset
1344 # minimal test that code to extract host from url works
044f9e71320f SF bug report #405939: wrong Host header with proxy
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 21962
diff changeset
1345 class HTTP11(HTTP):
044f9e71320f SF bug report #405939: wrong Host header with proxy
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 21962
diff changeset
1346 _http_vsn = 11
044f9e71320f SF bug report #405939: wrong Host header with proxy
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 21962
diff changeset
1347 _http_vsn_str = 'HTTP/1.1'
044f9e71320f SF bug report #405939: wrong Host header with proxy
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 21962
diff changeset
1348
044f9e71320f SF bug report #405939: wrong Host header with proxy
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 21962
diff changeset
1349 h = HTTP11('www.python.org')
044f9e71320f SF bug report #405939: wrong Host header with proxy
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 21962
diff changeset
1350 h.putrequest('GET', 'http://www.python.org/~jeremy/')
044f9e71320f SF bug report #405939: wrong Host header with proxy
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 21962
diff changeset
1351 h.endheaders()
044f9e71320f SF bug report #405939: wrong Host header with proxy
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 21962
diff changeset
1352 h.getreply()
044f9e71320f SF bug report #405939: wrong Host header with proxy
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 21962
diff changeset
1353 h.close()
044f9e71320f SF bug report #405939: wrong Host header with proxy
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 21962
diff changeset
1354
43008
4771639fc049 This contains a number of things:
Bill Janssen <janssen@parc.com>
parents: 42385
diff changeset
1355 try:
4771639fc049 This contains a number of things:
Bill Janssen <janssen@parc.com>
parents: 42385
diff changeset
1356 import ssl
4771639fc049 This contains a number of things:
Bill Janssen <janssen@parc.com>
parents: 42385
diff changeset
1357 except ImportError:
4771639fc049 This contains a number of things:
Bill Janssen <janssen@parc.com>
parents: 42385
diff changeset
1358 pass
4771639fc049 This contains a number of things:
Bill Janssen <janssen@parc.com>
parents: 42385
diff changeset
1359 else:
24263
e6033254df21 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 24262
diff changeset
1360
24168
fdd836575248 Fix for SF bug 579107.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24123
diff changeset
1361 for host, selector in (('sourceforge.net', '/projects/python'),
fdd836575248 Fix for SF bug 579107.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24123
diff changeset
1362 ):
fdd836575248 Fix for SF bug 579107.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24123
diff changeset
1363 print "https://%s%s" % (host, selector)
fdd836575248 Fix for SF bug 579107.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24123
diff changeset
1364 hs = HTTPS()
24262
3b571e32a6e4 Send HTTP requests with a single send() call instead of many.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24216
diff changeset
1365 hs.set_debuglevel(dl)
24168
fdd836575248 Fix for SF bug 579107.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24123
diff changeset
1366 hs.connect(host)
fdd836575248 Fix for SF bug 579107.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24123
diff changeset
1367 hs.putrequest('GET', selector)
fdd836575248 Fix for SF bug 579107.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24123
diff changeset
1368 hs.endheaders()
fdd836575248 Fix for SF bug 579107.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24123
diff changeset
1369 status, reason, headers = hs.getreply()
fdd836575248 Fix for SF bug 579107.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24123
diff changeset
1370 print 'status =', status
fdd836575248 Fix for SF bug 579107.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24123
diff changeset
1371 print 'reason =', reason
fdd836575248 Fix for SF bug 579107.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24123
diff changeset
1372 print "read", len(hs.getfile().read())
fdd836575248 Fix for SF bug 579107.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24123
diff changeset
1373 print
fdd836575248 Fix for SF bug 579107.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24123
diff changeset
1374 if headers:
fdd836575248 Fix for SF bug 579107.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24123
diff changeset
1375 for header in headers.headers: print header.strip()
fdd836575248 Fix for SF bug 579107.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24123
diff changeset
1376 print
1555
6d35308261f4 Towards a standard access mechanism for URLs.
Guido van Rossum <guido@python.org>
parents:
diff changeset
1377
6d35308261f4 Towards a standard access mechanism for URLs.
Guido van Rossum <guido@python.org>
parents:
diff changeset
1378 if __name__ == '__main__':
6787
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
1379 test()