Mercurial > cpython-lru_cache-weakref

annotate Lib/httplib.py @ 65035:87f4b32905df 2.7

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Cosmetic fix to use print function.
author Hirokazu Yamamoto <ocean-city@m2.ccsnet.ne.jp>
date Fri, 01 Oct 2010 10:48:47 +0000
parents 2c73e6ce6a75
children a748bfcb6973
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1 """HTTP/1.1 client library
6787
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
2
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
3 <intro stuff goes here>
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
4 <other stuff, too>
6787
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
5
39293
efbaec9674df Minor typo fixes
Andrew M. Kuchling <amk@amk.ca>
parents: 39190
diff changeset
6 HTTPConnection goes through a number of "states", which define when a client
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
7 may legally make another request or fetch the response for a particular
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
8 request. This diagram details these state transitions:
1555
6d35308261f4 Towards a standard access mechanism for URLs.
Guido van Rossum <guido@python.org>
parents:
diff changeset
9
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
10 (null)
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
11 |
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
12 | HTTPConnection()
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
13 v
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
14 Idle
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
15 |
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
16 | putrequest()
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
17 v
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
18 Request-started
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
19 |
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
20 | ( putheader() )* endheaders()
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
21 v
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
22 Request-sent
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
23 |
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
24 | response = getresponse()
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
25 v
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
26 Unread-response [Response-headers-read]
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
27 |\____________________
15845
210bbe53b6d4 No text file relying on significant trailing whitespace is robust under
Tim Peters <tim.peters@gmail.com>
parents: 15844
diff changeset
28 | |
210bbe53b6d4 No text file relying on significant trailing whitespace is robust under
Tim Peters <tim.peters@gmail.com>
parents: 15844
diff changeset
29 | response.read() | putrequest()
210bbe53b6d4 No text file relying on significant trailing whitespace is robust under
Tim Peters <tim.peters@gmail.com>
parents: 15844
diff changeset
30 v v
210bbe53b6d4 No text file relying on significant trailing whitespace is robust under
Tim Peters <tim.peters@gmail.com>
parents: 15844
diff changeset
31 Idle Req-started-unread-response
210bbe53b6d4 No text file relying on significant trailing whitespace is robust under
Tim Peters <tim.peters@gmail.com>
parents: 15844
diff changeset
32 ______/|
210bbe53b6d4 No text file relying on significant trailing whitespace is robust under
Tim Peters <tim.peters@gmail.com>
parents: 15844
diff changeset
33 / |
210bbe53b6d4 No text file relying on significant trailing whitespace is robust under
Tim Peters <tim.peters@gmail.com>
parents: 15844
diff changeset
34 response.read() | | ( putheader() )* endheaders()
210bbe53b6d4 No text file relying on significant trailing whitespace is robust under
Tim Peters <tim.peters@gmail.com>
parents: 15844
diff changeset
35 v v
210bbe53b6d4 No text file relying on significant trailing whitespace is robust under
Tim Peters <tim.peters@gmail.com>
parents: 15844
diff changeset
36 Request-started Req-sent-unread-response
210bbe53b6d4 No text file relying on significant trailing whitespace is robust under
Tim Peters <tim.peters@gmail.com>
parents: 15844
diff changeset
37 |
210bbe53b6d4 No text file relying on significant trailing whitespace is robust under
Tim Peters <tim.peters@gmail.com>
parents: 15844
diff changeset
38 | response.read()
210bbe53b6d4 No text file relying on significant trailing whitespace is robust under
Tim Peters <tim.peters@gmail.com>
parents: 15844
diff changeset
39 v
210bbe53b6d4 No text file relying on significant trailing whitespace is robust under
Tim Peters <tim.peters@gmail.com>
parents: 15844
diff changeset
40 Request-sent
6787
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
41
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
42 This diagram presents the following rules:
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
43 -- a second request may not be started until {response-headers-read}
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
44 -- a response [object] cannot be retrieved until {request-sent}
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
45 -- there is no differentiation between an unread response body and a
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
46 partially read response body
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
47
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
48 Note: this enforcement is applied by the HTTPConnection class. The
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
49 HTTPResponse class does not enforce this state machine, which
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
50 implies sophisticated clients may accelerate the request/response
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
51 pipeline. Caution should be taken, though: accelerating the states
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
52 beyond the above pattern may imply knowledge of the server's
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
53 connection-close behavior for certain requests. For example, it
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
54 is impossible to tell whether the server will close the connection
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
55 UNTIL the response headers have been read; this means that further
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
56 requests cannot be placed into the pipeline until it is known that
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
57 the server will NOT be closing the connection.
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
58
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
59 Logical State __state __response
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
60 ------------- ------- ----------
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
61 Idle _CS_IDLE None
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
62 Request-started _CS_REQ_STARTED None
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
63 Request-sent _CS_REQ_SENT None
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
64 Unread-response _CS_IDLE <response_class>
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
65 Req-started-unread-response _CS_REQ_STARTED <response_class>
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
66 Req-sent-unread-response _CS_REQ_SENT <response_class>
6787
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
67 """
1555
6d35308261f4 Towards a standard access mechanism for URLs.
Guido van Rossum <guido@python.org>
parents:
diff changeset
68
56285
e0f234116ab1 Issue #6790: Make it possible again to pass an `array.array` to
Antoine Pitrou <solipsis@pitrou.net>
parents: 55827
diff changeset
69 from array import array
63194
8072dfd97e1d Forward port r70643 (#5542) + part of r83120 (just remove the comment)
Victor Stinner <victor.stinner@haypocalc.com>
parents: 62064
diff changeset
70 import os
1555
6d35308261f4 Towards a standard access mechanism for URLs.
Guido van Rossum <guido@python.org>
parents:
diff changeset
71 import socket
48948
d563621d76db Move test.test_support.catch_warning() to the warnings module, rename it
Brett Cannon <bcannon@gmail.com>
parents: 48249
diff changeset
72 from sys import py3kwarning
22192
044f9e71320f SF bug report #405939: wrong Host header with proxy
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 21962
diff changeset
73 from urlparse import urlsplit
43161
9cfad9a5c83b deprecate use of FakeSocket
Bill Janssen <janssen@parc.com>
parents: 43093
diff changeset
74 import warnings
48948
d563621d76db Move test.test_support.catch_warning() to the warnings module, rename it
Brett Cannon <bcannon@gmail.com>
parents: 48249
diff changeset
75 with warnings.catch_warnings():
d563621d76db Move test.test_support.catch_warning() to the warnings module, rename it
Brett Cannon <bcannon@gmail.com>
parents: 48249
diff changeset
76 if py3kwarning:
d563621d76db Move test.test_support.catch_warning() to the warnings module, rename it
Brett Cannon <bcannon@gmail.com>
parents: 48249
diff changeset
77 warnings.filterwarnings("ignore", ".*mimetools has been removed",
d563621d76db Move test.test_support.catch_warning() to the warnings module, rename it
Brett Cannon <bcannon@gmail.com>
parents: 48249
diff changeset
78 DeprecationWarning)
48249
918a918a9996 Silence the DeprecationWarning raised in httplib when mimetools is imported.
Brett Cannon <bcannon@gmail.com>
parents: 47909
diff changeset
79 import mimetools
1555
6d35308261f4 Towards a standard access mechanism for URLs.
Guido van Rossum <guido@python.org>
parents:
diff changeset
80
11527
28b8585354de OpenSSL support. This is based on patches for a version of SSLeay by
Guido van Rossum <guido@python.org>
parents: 7122
diff changeset
81 try:
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
82 from cStringIO import StringIO
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
83 except ImportError:
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
84 from StringIO import StringIO
11527
28b8585354de OpenSSL support. This is based on patches for a version of SSLeay by
Guido van Rossum <guido@python.org>
parents: 7122
diff changeset
85
43093
1d57fb7efb15 Don't lie in __all__ attributes when SSL is not available: only add the SSL
Thomas Wouters <thomas@python.org>
parents: 43008
diff changeset
86 __all__ = ["HTTP", "HTTPResponse", "HTTPConnection",
18013
99aea6f5cebd more public symbols for __all__
Skip Montanaro <skip@pobox.com>
parents: 17482
diff changeset
87 "HTTPException", "NotConnected", "UnknownProtocol",
24042
63fbbed10714 Simplify HTTPSConnection constructor.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24038
diff changeset
88 "UnknownTransferEncoding", "UnimplementedFileMode",
63fbbed10714 Simplify HTTPSConnection constructor.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24038
diff changeset
89 "IncompleteRead", "InvalidURL", "ImproperConnectionState",
63fbbed10714 Simplify HTTPSConnection constructor.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24038
diff changeset
90 "CannotSendRequest", "CannotSendHeader", "ResponseNotReady",
36559
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
91 "BadStatusLine", "error", "responses"]
16160
46920b60cd33 added a few more __all__ lists
Skip Montanaro <skip@pobox.com>
parents: 15878
diff changeset
92
1555
6d35308261f4 Towards a standard access mechanism for URLs.
Guido van Rossum <guido@python.org>
parents:
diff changeset
93 HTTP_PORT = 80
11527
28b8585354de OpenSSL support. This is based on patches for a version of SSLeay by
Guido van Rossum <guido@python.org>
parents: 7122
diff changeset
94 HTTPS_PORT = 443
28b8585354de OpenSSL support. This is based on patches for a version of SSLeay by
Guido van Rossum <guido@python.org>
parents: 7122
diff changeset
95
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
96 _UNKNOWN = 'UNKNOWN'
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
97
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
98 # connection states
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
99 _CS_IDLE = 'Idle'
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
100 _CS_REQ_STARTED = 'Request-started'
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
101 _CS_REQ_SENT = 'Request-sent'
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
102
33506
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
103 # status codes
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
104 # informational
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
105 CONTINUE = 100
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
106 SWITCHING_PROTOCOLS = 101
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
107 PROCESSING = 102
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
108
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
109 # successful
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
110 OK = 200
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
111 CREATED = 201
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
112 ACCEPTED = 202
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
113 NON_AUTHORITATIVE_INFORMATION = 203
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
114 NO_CONTENT = 204
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
115 RESET_CONTENT = 205
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
116 PARTIAL_CONTENT = 206
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
117 MULTI_STATUS = 207
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
118 IM_USED = 226
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
119
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
120 # redirection
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
121 MULTIPLE_CHOICES = 300
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
122 MOVED_PERMANENTLY = 301
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
123 FOUND = 302
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
124 SEE_OTHER = 303
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
125 NOT_MODIFIED = 304
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
126 USE_PROXY = 305
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
127 TEMPORARY_REDIRECT = 307
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
128
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
129 # client error
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
130 BAD_REQUEST = 400
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
131 UNAUTHORIZED = 401
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
132 PAYMENT_REQUIRED = 402
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
133 FORBIDDEN = 403
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
134 NOT_FOUND = 404
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
135 METHOD_NOT_ALLOWED = 405
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
136 NOT_ACCEPTABLE = 406
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
137 PROXY_AUTHENTICATION_REQUIRED = 407
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
138 REQUEST_TIMEOUT = 408
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
139 CONFLICT = 409
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
140 GONE = 410
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
141 LENGTH_REQUIRED = 411
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
142 PRECONDITION_FAILED = 412
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
143 REQUEST_ENTITY_TOO_LARGE = 413
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
144 REQUEST_URI_TOO_LONG = 414
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
145 UNSUPPORTED_MEDIA_TYPE = 415
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
146 REQUESTED_RANGE_NOT_SATISFIABLE = 416
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
147 EXPECTATION_FAILED = 417
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
148 UNPROCESSABLE_ENTITY = 422
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
149 LOCKED = 423
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
150 FAILED_DEPENDENCY = 424
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
151 UPGRADE_REQUIRED = 426
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
152
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
153 # server error
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
154 INTERNAL_SERVER_ERROR = 500
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
155 NOT_IMPLEMENTED = 501
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
156 BAD_GATEWAY = 502
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
157 SERVICE_UNAVAILABLE = 503
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
158 GATEWAY_TIMEOUT = 504
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
159 HTTP_VERSION_NOT_SUPPORTED = 505
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
160 INSUFFICIENT_STORAGE = 507
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
161 NOT_EXTENDED = 510
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
162
36559
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
163 # Mapping status codes to official W3C names
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
164 responses = {
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
165 100: 'Continue',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
166 101: 'Switching Protocols',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
167
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
168 200: 'OK',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
169 201: 'Created',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
170 202: 'Accepted',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
171 203: 'Non-Authoritative Information',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
172 204: 'No Content',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
173 205: 'Reset Content',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
174 206: 'Partial Content',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
175
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
176 300: 'Multiple Choices',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
177 301: 'Moved Permanently',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
178 302: 'Found',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
179 303: 'See Other',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
180 304: 'Not Modified',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
181 305: 'Use Proxy',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
182 306: '(Unused)',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
183 307: 'Temporary Redirect',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
184
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
185 400: 'Bad Request',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
186 401: 'Unauthorized',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
187 402: 'Payment Required',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
188 403: 'Forbidden',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
189 404: 'Not Found',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
190 405: 'Method Not Allowed',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
191 406: 'Not Acceptable',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
192 407: 'Proxy Authentication Required',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
193 408: 'Request Timeout',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
194 409: 'Conflict',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
195 410: 'Gone',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
196 411: 'Length Required',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
197 412: 'Precondition Failed',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
198 413: 'Request Entity Too Large',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
199 414: 'Request-URI Too Long',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
200 415: 'Unsupported Media Type',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
201 416: 'Requested Range Not Satisfiable',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
202 417: 'Expectation Failed',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
203
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
204 500: 'Internal Server Error',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
205 501: 'Not Implemented',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
206 502: 'Bad Gateway',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
207 503: 'Service Unavailable',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
208 504: 'Gateway Timeout',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
209 505: 'HTTP Version Not Supported',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
210 }
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
211
35652
6a20831d357d bug [ 1296004 ] MemoryError in httplib
Georg Brandl <georg@python.org>
parents: 35158
diff changeset
212 # maximal amount of data to read at one time in _safe_read
6a20831d357d bug [ 1296004 ] MemoryError in httplib
Georg Brandl <georg@python.org>
parents: 35158
diff changeset
213 MAXAMOUNT = 1048576
6a20831d357d bug [ 1296004 ] MemoryError in httplib
Georg Brandl <georg@python.org>
parents: 35158
diff changeset
214
24123
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
215 class HTTPMessage(mimetools.Message):
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
216
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
217 def addheader(self, key, value):
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
218 """Add header for field key handling repeats."""
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
219 prev = self.dict.get(key)
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
220 if prev is None:
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
221 self.dict[key] = value
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
222 else:
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
223 combined = ", ".join((prev, value))
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
224 self.dict[key] = combined
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
225
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
226 def addcontinue(self, key, more):
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
227 """Add more field data from a continuation line."""
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
228 prev = self.dict[key]
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
229 self.dict[key] = prev + "\n " + more
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
230
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
231 def readheaders(self):
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
232 """Read header lines.
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
233
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
234 Read header lines up to the entirely blank line that terminates them.
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
235 The (normally blank) line that ends the headers is skipped, but not
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
236 included in the returned list. If a non-header line ends the headers,
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
237 (which is an error), an attempt is made to backspace over it; it is
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
238 never included in the returned list.
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
239
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
240 The variable self.status is set to the empty string if all went well,
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
241 otherwise it is an error message. The variable self.headers is a
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
242 completely uninterpreted list of lines contained in the header (so
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
243 printing them will reproduce the header exactly as it appears in the
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
244 file).
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
245
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
246 If multiple header fields with the same name occur, they are combined
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
247 according to the rules in RFC 2616 sec 4.2:
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
248
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
249 Appending each subsequent field-value to the first, each separated
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
250 by a comma. The order in which header fields with the same field-name
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
251 are received is significant to the interpretation of the combined
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
252 field value.
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
253 """
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
254 # XXX The implementation overrides the readheaders() method of
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
255 # rfc822.Message. The base class design isn't amenable to
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
256 # customized behavior here so the method here is a copy of the
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
257 # base class code with a few small changes.
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
258
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
259 self.dict = {}
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
260 self.unixfrom = ''
28083
29965d0d4650 Module review:
Raymond Hettinger <python@rcn.com>
parents: 28063
diff changeset
261 self.headers = hlist = []
24123
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
262 self.status = ''
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
263 headerseen = ""
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
264 firstline = 1
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
265 startofline = unread = tell = None
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
266 if hasattr(self.fp, 'unread'):
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
267 unread = self.fp.unread
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
268 elif self.seekable:
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
269 tell = self.fp.tell
28083
29965d0d4650 Module review:
Raymond Hettinger <python@rcn.com>
parents: 28063
diff changeset
270 while True:
24123
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
271 if tell:
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
272 try:
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
273 startofline = tell()
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
274 except IOError:
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
275 startofline = tell = None
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
276 self.seekable = 0
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
277 line = self.fp.readline()
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
278 if not line:
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
279 self.status = 'EOF in headers'
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
280 break
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
281 # Skip unix From name time lines
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
282 if firstline and line.startswith('From '):
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
283 self.unixfrom = self.unixfrom + line
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
284 continue
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
285 firstline = 0
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
286 if headerseen and line[0] in ' \t':
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
287 # XXX Not sure if continuation lines are handled properly
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
288 # for http and/or for repeating headers
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
289 # It's a continuation line.
28083
29965d0d4650 Module review:
Raymond Hettinger <python@rcn.com>
parents: 28063
diff changeset
290 hlist.append(line)
24123
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
291 self.addcontinue(headerseen, line.strip())
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
292 continue
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
293 elif self.iscomment(line):
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
294 # It's a comment. Ignore it.
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
295 continue
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
296 elif self.islast(line):
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
297 # Note! No pushback here! The delimiter line gets eaten.
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
298 break
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
299 headerseen = self.isheader(line)
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
300 if headerseen:
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
301 # It's a legal header line, save it.
28083
29965d0d4650 Module review:
Raymond Hettinger <python@rcn.com>
parents: 28063
diff changeset
302 hlist.append(line)
24123
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
303 self.addheader(headerseen, line[len(headerseen)+1:].strip())
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
304 continue
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
305 else:
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
306 # It's not a header line; throw it back and stop here.
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
307 if not self.dict:
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
308 self.status = 'No headers'
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
309 else:
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
310 self.status = 'Non-header line where header expected'
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
311 # Try to undo the read.
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
312 if unread:
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
313 unread(line)
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
314 elif tell:
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
315 self.fp.seek(startofline)
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
316 else:
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
317 self.status = self.status + '; bad seek'
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
318 break
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
319
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
320 class HTTPResponse:
24119
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
321
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
322 # strict: If true, raise BadStatusLine if the status line can't be
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
323 # parsed as a valid HTTP/1.0 or 1.1 status line. By default it is
24378
Skip Montanaro <skip@pobox.com>
parents: 24264
diff changeset
324 # false because it prevents clients from talking to HTTP/0.9
24119
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
325 # servers. Note that a response with a sufficiently corrupted
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
326 # status line will look like an HTTP/0.9 response.
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
327
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
328 # See RFC 2616 sec 19.6 and RFC 1945 sec 6 for details.
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
329
50734
d09d6fe31b61 Issue 4879: Allow buffering for HTTPResponse
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 50672
diff changeset
330 def __init__(self, sock, debuglevel=0, strict=0, method=None, buffering=False):
d09d6fe31b61 Issue 4879: Allow buffering for HTTPResponse
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 50672
diff changeset
331 if buffering:
d09d6fe31b61 Issue 4879: Allow buffering for HTTPResponse
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 50672
diff changeset
332 # The caller won't be using any sock.recv() calls, so buffering
55703
e471d6d32724 comment typo fix
Gregory P. Smith <greg@mad-scientist.com>
parents: 55702
diff changeset
333 # is fine and recommended for performance.
50734
d09d6fe31b61 Issue 4879: Allow buffering for HTTPResponse
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 50672
diff changeset
334 self.fp = sock.makefile('rb')
d09d6fe31b61 Issue 4879: Allow buffering for HTTPResponse
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 50672
diff changeset
335 else:
d09d6fe31b61 Issue 4879: Allow buffering for HTTPResponse
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 50672
diff changeset
336 # The buffer size is specified as zero, because the headers of
d09d6fe31b61 Issue 4879: Allow buffering for HTTPResponse
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 50672
diff changeset
337 # the response are read with readline(). If the reads were
d09d6fe31b61 Issue 4879: Allow buffering for HTTPResponse
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 50672
diff changeset
338 # buffered the readline() calls could consume some of the
d09d6fe31b61 Issue 4879: Allow buffering for HTTPResponse
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 50672
diff changeset
339 # response, which make be read via a recv() on the underlying
d09d6fe31b61 Issue 4879: Allow buffering for HTTPResponse
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 50672
diff changeset
340 # socket.
d09d6fe31b61 Issue 4879: Allow buffering for HTTPResponse
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 50672
diff changeset
341 self.fp = sock.makefile('rb', 0)
14672
1b20ad6eb501 Do not close socket when a Content-Length is 0. This make the
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14560
diff changeset
342 self.debuglevel = debuglevel
24119
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
343 self.strict = strict
28978
83fe0f5c9359 SF bug 622042: Don't expect response body from HEAD request.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 28671
diff changeset
344 self._method = method
11527
28b8585354de OpenSSL support. This is based on patches for a version of SSLeay by
Guido van Rossum <guido@python.org>
parents: 7122
diff changeset
345
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
346 self.msg = None
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
347
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
348 # from the Status-Line of the response
15868
3060cbfa01be Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15859
diff changeset
349 self.version = _UNKNOWN # HTTP-Version
3060cbfa01be Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15859
diff changeset
350 self.status = _UNKNOWN # Status-Code
3060cbfa01be Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15859
diff changeset
351 self.reason = _UNKNOWN # Reason-Phrase
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
352
15868
3060cbfa01be Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15859
diff changeset
353 self.chunked = _UNKNOWN # is "chunked" being used?
3060cbfa01be Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15859
diff changeset
354 self.chunk_left = _UNKNOWN # bytes left to read in current chunk
3060cbfa01be Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15859
diff changeset
355 self.length = _UNKNOWN # number of bytes left in response
3060cbfa01be Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15859
diff changeset
356 self.will_close = _UNKNOWN # conn will close at end of response
11527
28b8585354de OpenSSL support. This is based on patches for a version of SSLeay by
Guido van Rossum <guido@python.org>
parents: 7122
diff changeset
357
24038
9377158652dd Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23620
diff changeset
358 def _read_status(self):
24119
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
359 # Initialize with Simple-Response defaults
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
360 line = self.fp.readline()
14672
1b20ad6eb501 Do not close socket when a Content-Length is 0. This make the
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14560
diff changeset
361 if self.debuglevel > 0:
1b20ad6eb501 Do not close socket when a Content-Length is 0. This make the
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14560
diff changeset
362 print "reply:", repr(line)
29688
9fe67f7f3459 Fix sf bug 666219: assertion error in httplib.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 29584
diff changeset
363 if not line:
9fe67f7f3459 Fix sf bug 666219: assertion error in httplib.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 29584
diff changeset
364 # Presumably, the server closed the connection before
9fe67f7f3459 Fix sf bug 666219: assertion error in httplib.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 29584
diff changeset
365 # sending a valid response.
9fe67f7f3459 Fix sf bug 666219: assertion error in httplib.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 29584
diff changeset
366 raise BadStatusLine(line)
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
367 try:
15582
6179d4afa4e4 Get rid of string functions.
Guido van Rossum <guido@python.org>
parents: 15503
diff changeset
368 [version, status, reason] = line.split(None, 2)
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
369 except ValueError:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
370 try:
15582
6179d4afa4e4 Get rid of string functions.
Guido van Rossum <guido@python.org>
parents: 15503
diff changeset
371 [version, status] = line.split(None, 1)
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
372 reason = ""
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
373 except ValueError:
24119
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
374 # empty version will cause next test to fail and status
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
375 # will be treated as 0.9 response.
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
376 version = ""
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
377 if not version.startswith('HTTP/'):
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
378 if self.strict:
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
379 self.close()
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
380 raise BadStatusLine(line)
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
381 else:
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
382 # assume it's a Simple-Response from an 0.9 server
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
383 self.fp = LineAndFileWrapper(line, self.fp)
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
384 return "HTTP/0.9", 200, ""
11527
28b8585354de OpenSSL support. This is based on patches for a version of SSLeay by
Guido van Rossum <guido@python.org>
parents: 7122
diff changeset
385
17482
2265032efc9f SF patch #405845 by Martin von Löwis
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 16400
diff changeset
386 # The status code is a three-digit number
2265032efc9f SF patch #405845 by Martin von Löwis
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 16400
diff changeset
387 try:
24038
9377158652dd Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23620
diff changeset
388 status = int(status)
17482
2265032efc9f SF patch #405845 by Martin von Löwis
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 16400
diff changeset
389 if status < 100 or status > 999:
2265032efc9f SF patch #405845 by Martin von Löwis
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 16400
diff changeset
390 raise BadStatusLine(line)
2265032efc9f SF patch #405845 by Martin von Löwis
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 16400
diff changeset
391 except ValueError:
2265032efc9f SF patch #405845 by Martin von Löwis
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 16400
diff changeset
392 raise BadStatusLine(line)
24038
9377158652dd Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23620
diff changeset
393 return version, status, reason
9377158652dd Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23620
diff changeset
394
24216
c8a19a428358 Change _begin() back to begin().
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24168
diff changeset
395 def begin(self):
24038
9377158652dd Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23620
diff changeset
396 if self.msg is not None:
9377158652dd Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23620
diff changeset
397 # we've already started reading the response
9377158652dd Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23620
diff changeset
398 return
9377158652dd Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23620
diff changeset
399
9377158652dd Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23620
diff changeset
400 # read until we get a non-100 response
28083
29965d0d4650 Module review:
Raymond Hettinger <python@rcn.com>
parents: 28063
diff changeset
401 while True:
24038
9377158652dd Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23620
diff changeset
402 version, status, reason = self._read_status()
33506
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
403 if status != CONTINUE:
24038
9377158652dd Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23620
diff changeset
404 break
9377158652dd Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23620
diff changeset
405 # skip the header from the 100 response
28083
29965d0d4650 Module review:
Raymond Hettinger <python@rcn.com>
parents: 28063
diff changeset
406 while True:
24038
9377158652dd Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23620
diff changeset
407 skip = self.fp.readline().strip()
9377158652dd Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23620
diff changeset
408 if not skip:
9377158652dd Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23620
diff changeset
409 break
9377158652dd Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23620
diff changeset
410 if self.debuglevel > 0:
9377158652dd Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23620
diff changeset
411 print "header:", skip
24263
e6033254df21 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 24262
diff changeset
412
24038
9377158652dd Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23620
diff changeset
413 self.status = status
15582
6179d4afa4e4 Get rid of string functions.
Guido van Rossum <guido@python.org>
parents: 15503
diff changeset
414 self.reason = reason.strip()
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
415 if version == 'HTTP/1.0':
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
416 self.version = 10
15182
b3387eeba68d If the status line is invalid, assume it is a pre-1.0 response. The
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14740
diff changeset
417 elif version.startswith('HTTP/1.'):
15868
3060cbfa01be Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15859
diff changeset
418 self.version = 11 # use HTTP/1.1 code for HTTP/1.x where x>=1
15182
b3387eeba68d If the status line is invalid, assume it is a pre-1.0 response. The
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14740
diff changeset
419 elif version == 'HTTP/0.9':
b3387eeba68d If the status line is invalid, assume it is a pre-1.0 response. The
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14740
diff changeset
420 self.version = 9
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
421 else:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
422 raise UnknownProtocol(version)
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
423
15182
b3387eeba68d If the status line is invalid, assume it is a pre-1.0 response. The
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14740
diff changeset
424 if self.version == 9:
35158
daebfde0e1cf backport bug [ 1155638 ] self.length shield exception in httplib
Georg Brandl <georg@python.org>
parents: 33933
diff changeset
425 self.length = None
24038
9377158652dd Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23620
diff changeset
426 self.chunked = 0
24119
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
427 self.will_close = 1
24123
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
428 self.msg = HTTPMessage(StringIO())
15182
b3387eeba68d If the status line is invalid, assume it is a pre-1.0 response. The
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14740
diff changeset
429 return
b3387eeba68d If the status line is invalid, assume it is a pre-1.0 response. The
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14740
diff changeset
430
24123
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
431 self.msg = HTTPMessage(self.fp, 0)
14672
1b20ad6eb501 Do not close socket when a Content-Length is 0. This make the
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14560
diff changeset
432 if self.debuglevel > 0:
1b20ad6eb501 Do not close socket when a Content-Length is 0. This make the
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14560
diff changeset
433 for hdr in self.msg.headers:
1b20ad6eb501 Do not close socket when a Content-Length is 0. This make the
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14560
diff changeset
434 print "header:", hdr,
1555
6d35308261f4 Towards a standard access mechanism for URLs.
Guido van Rossum <guido@python.org>
parents:
diff changeset
435
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
436 # don't let the msg keep an fp
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
437 self.msg.fp = None
6787
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
438
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
439 # are we using the chunked-style of transfer encoding?
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
440 tr_enc = self.msg.getheader('transfer-encoding')
25107
0ecb3cabf40b SF bug [ 600488 ] Robustness tweak to httplib.py
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24674
diff changeset
441 if tr_enc and tr_enc.lower() == "chunked":
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
442 self.chunked = 1
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
443 self.chunk_left = None
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
444 else:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
445 self.chunked = 0
1555
6d35308261f4 Towards a standard access mechanism for URLs.
Guido van Rossum <guido@python.org>
parents:
diff changeset
446
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
447 # will the connection close at the end of the response?
26187
ae169f5b94e0 Fix SF bug #637789: Handle Proxy-Connection header.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 26126
diff changeset
448 self.will_close = self._check_close()
11527
28b8585354de OpenSSL support. This is based on patches for a version of SSLeay by
Guido van Rossum <guido@python.org>
parents: 7122
diff changeset
449
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
450 # do we have a Content-Length?
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
451 # NOTE: RFC 2616, S4.4, #3 says we ignore this if tr_enc is "chunked"
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
452 length = self.msg.getheader('content-length')
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
453 if length and not self.chunked:
14560
eeb8ee488106 cope with weird Content-Length values returned from servers by
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14140
diff changeset
454 try:
eeb8ee488106 cope with weird Content-Length values returned from servers by
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14140
diff changeset
455 self.length = int(length)
eeb8ee488106 cope with weird Content-Length values returned from servers by
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14140
diff changeset
456 except ValueError:
eeb8ee488106 cope with weird Content-Length values returned from servers by
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14140
diff changeset
457 self.length = None
45494
7efa4d1bced3 #1627: httplib now ignores negative Content-Length headers.
Georg Brandl <georg@python.org>
parents: 45493
diff changeset
458 else:
7efa4d1bced3 #1627: httplib now ignores negative Content-Length headers.
Georg Brandl <georg@python.org>
parents: 45493
diff changeset
459 if self.length < 0: # ignore nonsensical negative lengths
7efa4d1bced3 #1627: httplib now ignores negative Content-Length headers.
Georg Brandl <georg@python.org>
parents: 45493
diff changeset
460 self.length = None
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
461 else:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
462 self.length = None
11527
28b8585354de OpenSSL support. This is based on patches for a version of SSLeay by
Guido van Rossum <guido@python.org>
parents: 7122
diff changeset
463
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
464 # does the body have a fixed length? (of zero)
33506
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
465 if (status == NO_CONTENT or status == NOT_MODIFIED or
28978
83fe0f5c9359 SF bug 622042: Don't expect response body from HEAD request.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 28671
diff changeset
466 100 <= status < 200 or # 1xx codes
83fe0f5c9359 SF bug 622042: Don't expect response body from HEAD request.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 28671
diff changeset
467 self._method == 'HEAD'):
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
468 self.length = 0
1555
6d35308261f4 Towards a standard access mechanism for URLs.
Guido van Rossum <guido@python.org>
parents:
diff changeset
469
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
470 # if the connection remains open, and we aren't using chunked, and
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
471 # a content-length was not provided, then assume that the connection
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
472 # WILL close.
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
473 if not self.will_close and \
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
474 not self.chunked and \
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
475 self.length is None:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
476 self.will_close = 1
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
477
26187
ae169f5b94e0 Fix SF bug #637789: Handle Proxy-Connection header.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 26126
diff changeset
478 def _check_close(self):
33001
4fd59651fe3f SF bug 874842 and patch 997626: httplib bugs
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 32252
diff changeset
479 conn = self.msg.getheader('connection')
26187
ae169f5b94e0 Fix SF bug #637789: Handle Proxy-Connection header.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 26126
diff changeset
480 if self.version == 11:
ae169f5b94e0 Fix SF bug #637789: Handle Proxy-Connection header.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 26126
diff changeset
481 # An HTTP/1.1 proxy is assumed to stay open unless
ae169f5b94e0 Fix SF bug #637789: Handle Proxy-Connection header.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 26126
diff changeset
482 # explicitly closed.
ae169f5b94e0 Fix SF bug #637789: Handle Proxy-Connection header.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 26126
diff changeset
483 conn = self.msg.getheader('connection')
31967
3d046d063a84 Replace str.find()!=1 with the more readable "in" operator.
Raymond Hettinger <python@rcn.com>
parents: 31389
diff changeset
484 if conn and "close" in conn.lower():
26187
ae169f5b94e0 Fix SF bug #637789: Handle Proxy-Connection header.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 26126
diff changeset
485 return True
ae169f5b94e0 Fix SF bug #637789: Handle Proxy-Connection header.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 26126
diff changeset
486 return False
ae169f5b94e0 Fix SF bug #637789: Handle Proxy-Connection header.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 26126
diff changeset
487
33001
4fd59651fe3f SF bug 874842 and patch 997626: httplib bugs
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 32252
diff changeset
488 # Some HTTP/1.0 implementations have support for persistent
4fd59651fe3f SF bug 874842 and patch 997626: httplib bugs
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 32252
diff changeset
489 # connections, using rules different than HTTP/1.1.
26187
ae169f5b94e0 Fix SF bug #637789: Handle Proxy-Connection header.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 26126
diff changeset
490
44218
77cb8ff34912 Fix tpyo.
Georg Brandl <georg@python.org>
parents: 43584
diff changeset
491 # For older HTTP, Keep-Alive indicates persistent connection.
26187
ae169f5b94e0 Fix SF bug #637789: Handle Proxy-Connection header.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 26126
diff changeset
492 if self.msg.getheader('keep-alive'):
ae169f5b94e0 Fix SF bug #637789: Handle Proxy-Connection header.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 26126
diff changeset
493 return False
26387
fcf9734e1021 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 26188
diff changeset
494
33001
4fd59651fe3f SF bug 874842 and patch 997626: httplib bugs
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 32252
diff changeset
495 # At least Akamai returns a "Connection: Keep-Alive" header,
4fd59651fe3f SF bug 874842 and patch 997626: httplib bugs
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 32252
diff changeset
496 # which was supposed to be sent by the client.
4fd59651fe3f SF bug 874842 and patch 997626: httplib bugs
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 32252
diff changeset
497 if conn and "keep-alive" in conn.lower():
4fd59651fe3f SF bug 874842 and patch 997626: httplib bugs
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 32252
diff changeset
498 return False
4fd59651fe3f SF bug 874842 and patch 997626: httplib bugs
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 32252
diff changeset
499
26187
ae169f5b94e0 Fix SF bug #637789: Handle Proxy-Connection header.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 26126
diff changeset
500 # Proxy-Connection is a netscape hack.
ae169f5b94e0 Fix SF bug #637789: Handle Proxy-Connection header.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 26126
diff changeset
501 pconn = self.msg.getheader('proxy-connection')
31967
3d046d063a84 Replace str.find()!=1 with the more readable "in" operator.
Raymond Hettinger <python@rcn.com>
parents: 31389
diff changeset
502 if pconn and "keep-alive" in pconn.lower():
26187
ae169f5b94e0 Fix SF bug #637789: Handle Proxy-Connection header.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 26126
diff changeset
503 return False
ae169f5b94e0 Fix SF bug #637789: Handle Proxy-Connection header.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 26126
diff changeset
504
ae169f5b94e0 Fix SF bug #637789: Handle Proxy-Connection header.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 26126
diff changeset
505 # otherwise, assume it will close
ae169f5b94e0 Fix SF bug #637789: Handle Proxy-Connection header.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 26126
diff changeset
506 return True
ae169f5b94e0 Fix SF bug #637789: Handle Proxy-Connection header.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 26126
diff changeset
507
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
508 def close(self):
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
509 if self.fp:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
510 self.fp.close()
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
511 self.fp = None
11527
28b8585354de OpenSSL support. This is based on patches for a version of SSLeay by
Guido van Rossum <guido@python.org>
parents: 7122
diff changeset
512
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
513 def isclosed(self):
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
514 # NOTE: it is possible that we will not ever call self.close(). This
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
515 # case occurs when will_close is TRUE, length is None, and we
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
516 # read up to the last byte, but NOT past it.
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
517 #
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
518 # IMPLIES: if will_close is FALSE, then self.close() will ALWAYS be
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
519 # called, meaning self.isclosed() is meaningful.
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
520 return self.fp is None
6787
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
521
33001
4fd59651fe3f SF bug 874842 and patch 997626: httplib bugs
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 32252
diff changeset
522 # XXX It would be nice to have readline and __iter__ for this, too.
4fd59651fe3f SF bug 874842 and patch 997626: httplib bugs
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 32252
diff changeset
523
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
524 def read(self, amt=None):
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
525 if self.fp is None:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
526 return ''
1555
6d35308261f4 Towards a standard access mechanism for URLs.
Guido van Rossum <guido@python.org>
parents:
diff changeset
527
61056
57548cce1ea0 Fixed Issue6312 - httplib fails with HEAD requests to pages with "transfer-encoding: chunked"
Senthil Kumaran <orsenthil@gmail.com>
parents: 59107
diff changeset
528 if self._method == 'HEAD':
62064
3b67fa3585f0 Fix issue6312 - close the resp object for HEAD response.
Senthil Kumaran <orsenthil@gmail.com>
parents: 61056
diff changeset
529 self.close()
61056
57548cce1ea0 Fixed Issue6312 - httplib fails with HEAD requests to pages with "transfer-encoding: chunked"
Senthil Kumaran <orsenthil@gmail.com>
parents: 59107
diff changeset
530 return ''
57548cce1ea0 Fixed Issue6312 - httplib fails with HEAD requests to pages with "transfer-encoding: chunked"
Senthil Kumaran <orsenthil@gmail.com>
parents: 59107
diff changeset
531
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
532 if self.chunked:
25112
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
533 return self._read_chunked(amt)
26126
98f3fd1ed42f Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 25112
diff changeset
534
25112
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
535 if amt is None:
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
536 # unbounded read
33933
eb231544c1ad Fix for SF bug 988120 via patch 1061941.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 33761
diff changeset
537 if self.length is None:
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
538 s = self.fp.read()
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
539 else:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
540 s = self._safe_read(self.length)
33933
eb231544c1ad Fix for SF bug 988120 via patch 1061941.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 33761
diff changeset
541 self.length = 0
15868
3060cbfa01be Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15859
diff changeset
542 self.close() # we read everything
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
543 return s
1555
6d35308261f4 Towards a standard access mechanism for URLs.
Guido van Rossum <guido@python.org>
parents:
diff changeset
544
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
545 if self.length is not None:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
546 if amt > self.length:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
547 # clip the read to the "end of response"
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
548 amt = self.length
1555
6d35308261f4 Towards a standard access mechanism for URLs.
Guido van Rossum <guido@python.org>
parents:
diff changeset
549
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
550 # we do not use _safe_read() here because this may be a .will_close
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
551 # connection, and the user is reading more bytes than will be provided
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
552 # (for example, reading in 1k chunks)
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
553 s = self.fp.read(amt)
33933
eb231544c1ad Fix for SF bug 988120 via patch 1061941.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 33761
diff changeset
554 if self.length is not None:
eb231544c1ad Fix for SF bug 988120 via patch 1061941.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 33761
diff changeset
555 self.length -= len(s)
43584
da44334ebea2 Issue #1580738. When HTTPConnection reads the whole stream with read(),
Facundo Batista <facundobatista@gmail.com>
parents: 43325
diff changeset
556 if not self.length:
da44334ebea2 Issue #1580738. When HTTPConnection reads the whole stream with read(),
Facundo Batista <facundobatista@gmail.com>
parents: 43325
diff changeset
557 self.close()
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
558 return s
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
559
25112
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
560 def _read_chunked(self, amt):
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
561 assert self.chunked != _UNKNOWN
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
562 chunk_left = self.chunk_left
55827
77db25c7526a Fixes issue #6838: use a list to accumulate the value instead of repeatedly concatenating strings.
Chris Withers <chris@simplistix.co.uk>
parents: 55703
diff changeset
563 value = []
28083
29965d0d4650 Module review:
Raymond Hettinger <python@rcn.com>
parents: 28063
diff changeset
564 while True:
25112
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
565 if chunk_left is None:
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
566 line = self.fp.readline()
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
567 i = line.find(';')
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
568 if i >= 0:
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
569 line = line[:i] # strip chunk-extensions
45493
21a5e9a5483a #900744: If an invalid chunked-encoding header is sent by a server,
Georg Brandl <georg@python.org>
parents: 45468
diff changeset
570 try:
21a5e9a5483a #900744: If an invalid chunked-encoding header is sent by a server,
Georg Brandl <georg@python.org>
parents: 45468
diff changeset
571 chunk_left = int(line, 16)
21a5e9a5483a #900744: If an invalid chunked-encoding header is sent by a server,
Georg Brandl <georg@python.org>
parents: 45468
diff changeset
572 except ValueError:
21a5e9a5483a #900744: If an invalid chunked-encoding header is sent by a server,
Georg Brandl <georg@python.org>
parents: 45468
diff changeset
573 # close the connection as protocol synchronisation is
21a5e9a5483a #900744: If an invalid chunked-encoding header is sent by a server,
Georg Brandl <georg@python.org>
parents: 45468
diff changeset
574 # probably lost
21a5e9a5483a #900744: If an invalid chunked-encoding header is sent by a server,
Georg Brandl <georg@python.org>
parents: 45468
diff changeset
575 self.close()
55827
77db25c7526a Fixes issue #6838: use a list to accumulate the value instead of repeatedly concatenating strings.
Chris Withers <chris@simplistix.co.uk>
parents: 55703
diff changeset
576 raise IncompleteRead(''.join(value))
25112
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
577 if chunk_left == 0:
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
578 break
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
579 if amt is None:
55827
77db25c7526a Fixes issue #6838: use a list to accumulate the value instead of repeatedly concatenating strings.
Chris Withers <chris@simplistix.co.uk>
parents: 55703
diff changeset
580 value.append(self._safe_read(chunk_left))
25112
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
581 elif amt < chunk_left:
55827
77db25c7526a Fixes issue #6838: use a list to accumulate the value instead of repeatedly concatenating strings.
Chris Withers <chris@simplistix.co.uk>
parents: 55703
diff changeset
582 value.append(self._safe_read(amt))
25112
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
583 self.chunk_left = chunk_left - amt
55827
77db25c7526a Fixes issue #6838: use a list to accumulate the value instead of repeatedly concatenating strings.
Chris Withers <chris@simplistix.co.uk>
parents: 55703
diff changeset
584 return ''.join(value)
25112
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
585 elif amt == chunk_left:
55827
77db25c7526a Fixes issue #6838: use a list to accumulate the value instead of repeatedly concatenating strings.
Chris Withers <chris@simplistix.co.uk>
parents: 55703
diff changeset
586 value.append(self._safe_read(amt))
25112
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
587 self._safe_read(2) # toss the CRLF at the end of the chunk
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
588 self.chunk_left = None
55827
77db25c7526a Fixes issue #6838: use a list to accumulate the value instead of repeatedly concatenating strings.
Chris Withers <chris@simplistix.co.uk>
parents: 55703
diff changeset
589 return ''.join(value)
25112
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
590 else:
55827
77db25c7526a Fixes issue #6838: use a list to accumulate the value instead of repeatedly concatenating strings.
Chris Withers <chris@simplistix.co.uk>
parents: 55703
diff changeset
591 value.append(self._safe_read(chunk_left))
25112
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
592 amt -= chunk_left
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
593
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
594 # we read the whole chunk, get another
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
595 self._safe_read(2) # toss the CRLF at the end of the chunk
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
596 chunk_left = None
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
597
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
598 # read and discard trailer up to the CRLF terminator
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
599 ### note: we shouldn't have any trailers!
28083
29965d0d4650 Module review:
Raymond Hettinger <python@rcn.com>
parents: 28063
diff changeset
600 while True:
25112
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
601 line = self.fp.readline()
45298
8eb8e9a3260d Patch #1966: Break infinite loop in httplib when the servers
Martin v. Löwis <martin@v.loewis.de>
parents: 44218
diff changeset
602 if not line:
8eb8e9a3260d Patch #1966: Break infinite loop in httplib when the servers
Martin v. Löwis <martin@v.loewis.de>
parents: 44218
diff changeset
603 # a vanishingly small number of sites EOF without
8eb8e9a3260d Patch #1966: Break infinite loop in httplib when the servers
Martin v. Löwis <martin@v.loewis.de>
parents: 44218
diff changeset
604 # sending the trailer
8eb8e9a3260d Patch #1966: Break infinite loop in httplib when the servers
Martin v. Löwis <martin@v.loewis.de>
parents: 44218
diff changeset
605 break
25112
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
606 if line == '\r\n':
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
607 break
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
608
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
609 # we read everything; close the "file"
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
610 self.close()
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
611
55827
77db25c7526a Fixes issue #6838: use a list to accumulate the value instead of repeatedly concatenating strings.
Chris Withers <chris@simplistix.co.uk>
parents: 55703
diff changeset
612 return ''.join(value)
26126
98f3fd1ed42f Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 25112
diff changeset
613
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
614 def _safe_read(self, amt):
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
615 """Read the number of bytes requested, compensating for partial reads.
11527
28b8585354de OpenSSL support. This is based on patches for a version of SSLeay by
Guido van Rossum <guido@python.org>
parents: 7122
diff changeset
616
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
617 Normally, we have a blocking socket, but a read() can be interrupted
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
618 by a signal (resulting in a partial read).
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
619
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
620 Note that we cannot distinguish between EOF and an interrupt when zero
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
621 bytes have been read. IncompleteRead() will be raised in this
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
622 situation.
11527
28b8585354de OpenSSL support. This is based on patches for a version of SSLeay by
Guido van Rossum <guido@python.org>
parents: 7122
diff changeset
623
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
624 This function should be used when <amt> bytes "should" be present for
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
625 reading. If the bytes are truly not available (due to EOF), then the
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
626 IncompleteRead exception can be used to detect the problem.
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
627 """
55650
1377a8c26792 Force the http connection to close after any request returned when
Gregory P. Smith <greg@mad-scientist.com>
parents: 54298
diff changeset
628 # NOTE(gps): As of svn r74426 socket._fileobject.read(x) will never
1377a8c26792 Force the http connection to close after any request returned when
Gregory P. Smith <greg@mad-scientist.com>
parents: 54298
diff changeset
629 # return less than x bytes unless EOF is encountered. It now handles
1377a8c26792 Force the http connection to close after any request returned when
Gregory P. Smith <greg@mad-scientist.com>
parents: 54298
diff changeset
630 # signal interruptions (socket.error EINTR) internally. This code
1377a8c26792 Force the http connection to close after any request returned when
Gregory P. Smith <greg@mad-scientist.com>
parents: 54298
diff changeset
631 # never caught that exception anyways. It seems largely pointless.
1377a8c26792 Force the http connection to close after any request returned when
Gregory P. Smith <greg@mad-scientist.com>
parents: 54298
diff changeset
632 # self.fp.read(amt) will work fine.
35652
6a20831d357d bug [ 1296004 ] MemoryError in httplib
Georg Brandl <georg@python.org>
parents: 35158
diff changeset
633 s = []
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
634 while amt > 0:
35652
6a20831d357d bug [ 1296004 ] MemoryError in httplib
Georg Brandl <georg@python.org>
parents: 35158
diff changeset
635 chunk = self.fp.read(min(amt, MAXAMOUNT))
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
636 if not chunk:
51951
87115c727102 give httplib.IncompleteRead a more sane repr #4308
Benjamin Peterson <benjamin@python.org>
parents: 50734
diff changeset
637 raise IncompleteRead(''.join(s), amt)
35652
6a20831d357d bug [ 1296004 ] MemoryError in httplib
Georg Brandl <georg@python.org>
parents: 35158
diff changeset
638 s.append(chunk)
28083
29965d0d4650 Module review:
Raymond Hettinger <python@rcn.com>
parents: 28063
diff changeset
639 amt -= len(chunk)
35652
6a20831d357d bug [ 1296004 ] MemoryError in httplib
Georg Brandl <georg@python.org>
parents: 35158
diff changeset
640 return ''.join(s)
11527
28b8585354de OpenSSL support. This is based on patches for a version of SSLeay by
Guido van Rossum <guido@python.org>
parents: 7122
diff changeset
641
64862
2c73e6ce6a75 Fix Issue1327971: HTTPResponse should expose a proper fileno attribute
Senthil Kumaran <orsenthil@gmail.com>
parents: 63194
diff changeset
642 def fileno(self):
2c73e6ce6a75 Fix Issue1327971: HTTPResponse should expose a proper fileno attribute
Senthil Kumaran <orsenthil@gmail.com>
parents: 63194
diff changeset
643 return self.fp.fileno()
2c73e6ce6a75 Fix Issue1327971: HTTPResponse should expose a proper fileno attribute
Senthil Kumaran <orsenthil@gmail.com>
parents: 63194
diff changeset
644
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
645 def getheader(self, name, default=None):
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
646 if self.msg is None:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
647 raise ResponseNotReady()
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
648 return self.msg.getheader(name, default)
11527
28b8585354de OpenSSL support. This is based on patches for a version of SSLeay by
Guido van Rossum <guido@python.org>
parents: 7122
diff changeset
649
33173
e05e9d05830d Patch #800236: add HTTPResponse.getheaders().
Martin v. Löwis <martin@v.loewis.de>
parents: 33001
diff changeset
650 def getheaders(self):
e05e9d05830d Patch #800236: add HTTPResponse.getheaders().
Martin v. Löwis <martin@v.loewis.de>
parents: 33001
diff changeset
651 """Return list of (header, value) tuples."""
e05e9d05830d Patch #800236: add HTTPResponse.getheaders().
Martin v. Löwis <martin@v.loewis.de>
parents: 33001
diff changeset
652 if self.msg is None:
e05e9d05830d Patch #800236: add HTTPResponse.getheaders().
Martin v. Löwis <martin@v.loewis.de>
parents: 33001
diff changeset
653 raise ResponseNotReady()
e05e9d05830d Patch #800236: add HTTPResponse.getheaders().
Martin v. Löwis <martin@v.loewis.de>
parents: 33001
diff changeset
654 return self.msg.items()
e05e9d05830d Patch #800236: add HTTPResponse.getheaders().
Martin v. Löwis <martin@v.loewis.de>
parents: 33001
diff changeset
655
1555
6d35308261f4 Towards a standard access mechanism for URLs.
Guido van Rossum <guido@python.org>
parents:
diff changeset
656
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
657 class HTTPConnection:
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
658
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
659 _http_vsn = 11
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
660 _http_vsn_str = 'HTTP/1.1'
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
661
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
662 response_class = HTTPResponse
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
663 default_port = HTTP_PORT
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
664 auto_open = 1
14672
1b20ad6eb501 Do not close socket when a Content-Length is 0. This make the
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14560
diff changeset
665 debuglevel = 0
24119
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
666 strict = 0
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
667
47372
e2c5ae30cffe Fixed the semantic of timeout for socket.create_connection and
Facundo Batista <facundobatista@gmail.com>
parents: 45494
diff changeset
668 def __init__(self, host, port=None, strict=None,
58113
a19ad299095f issue3972: HTTPConnection and HTTPSConnection now support a
Gregory P. Smith <greg@mad-scientist.com>
parents: 57799
diff changeset
669 timeout=socket._GLOBAL_DEFAULT_TIMEOUT, source_address=None):
41288
a73aa0b821ca Added a 'create_connect()' function to socket.py, which creates a
Facundo Batista <facundobatista@gmail.com>
parents: 41285
diff changeset
670 self.timeout = timeout
58113
a19ad299095f issue3972: HTTPConnection and HTTPSConnection now support a
Gregory P. Smith <greg@mad-scientist.com>
parents: 57799
diff changeset
671 self.source_address = source_address
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
672 self.sock = None
24262
3b571e32a6e4 Send HTTP requests with a single send() call instead of many.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24216
diff changeset
673 self._buffer = []
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
674 self.__response = None
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
675 self.__state = _CS_IDLE
28978
83fe0f5c9359 SF bug 622042: Don't expect response body from HEAD request.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 28671
diff changeset
676 self._method = None
54298
e7b4a443f764 Fixed Issue1424152, urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 51951
diff changeset
677 self._tunnel_host = None
e7b4a443f764 Fixed Issue1424152, urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 51951
diff changeset
678 self._tunnel_port = None
57799
10970f64ba33 Fix for issue 7291 - urllib2 cannot handle https with proxy requiring auth
Senthil Kumaran <orsenthil@gmail.com>
parents: 56528
diff changeset
679 self._tunnel_headers = {}
24263
e6033254df21 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 24262
diff changeset
680
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
681 self._set_hostport(host, port)
24119
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
682 if strict is not None:
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
683 self.strict = strict
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
684
57799
10970f64ba33 Fix for issue 7291 - urllib2 cannot handle https with proxy requiring auth
Senthil Kumaran <orsenthil@gmail.com>
parents: 56528
diff changeset
685 def set_tunnel(self, host, port=None, headers=None):
10970f64ba33 Fix for issue 7291 - urllib2 cannot handle https with proxy requiring auth
Senthil Kumaran <orsenthil@gmail.com>
parents: 56528
diff changeset
686 """ Sets up the host and the port for the HTTP CONNECT Tunnelling.
10970f64ba33 Fix for issue 7291 - urllib2 cannot handle https with proxy requiring auth
Senthil Kumaran <orsenthil@gmail.com>
parents: 56528
diff changeset
687
10970f64ba33 Fix for issue 7291 - urllib2 cannot handle https with proxy requiring auth
Senthil Kumaran <orsenthil@gmail.com>
parents: 56528
diff changeset
688 The headers argument should be a mapping of extra HTTP headers
10970f64ba33 Fix for issue 7291 - urllib2 cannot handle https with proxy requiring auth
Senthil Kumaran <orsenthil@gmail.com>
parents: 56528
diff changeset
689 to send with the CONNECT request.
10970f64ba33 Fix for issue 7291 - urllib2 cannot handle https with proxy requiring auth
Senthil Kumaran <orsenthil@gmail.com>
parents: 56528
diff changeset
690 """
54298
e7b4a443f764 Fixed Issue1424152, urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 51951
diff changeset
691 self._tunnel_host = host
e7b4a443f764 Fixed Issue1424152, urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 51951
diff changeset
692 self._tunnel_port = port
57799
10970f64ba33 Fix for issue 7291 - urllib2 cannot handle https with proxy requiring auth
Senthil Kumaran <orsenthil@gmail.com>
parents: 56528
diff changeset
693 if headers:
10970f64ba33 Fix for issue 7291 - urllib2 cannot handle https with proxy requiring auth
Senthil Kumaran <orsenthil@gmail.com>
parents: 56528
diff changeset
694 self._tunnel_headers = headers
10970f64ba33 Fix for issue 7291 - urllib2 cannot handle https with proxy requiring auth
Senthil Kumaran <orsenthil@gmail.com>
parents: 56528
diff changeset
695 else:
10970f64ba33 Fix for issue 7291 - urllib2 cannot handle https with proxy requiring auth
Senthil Kumaran <orsenthil@gmail.com>
parents: 56528
diff changeset
696 self._tunnel_headers.clear()
54298
e7b4a443f764 Fixed Issue1424152, urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 51951
diff changeset
697
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
698 def _set_hostport(self, host, port):
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
699 if port is None:
33488
71ed4b35fdc4 Search from the end of the host/port combination to find the colon which
Skip Montanaro <skip@pobox.com>
parents: 33173
diff changeset
700 i = host.rfind(':')
33491
0c299cabf4f0 missed the obvious test case and corresponding fix
Skip Montanaro <skip@pobox.com>
parents: 33488
diff changeset
701 j = host.rfind(']') # ipv6 addresses have [...]
0c299cabf4f0 missed the obvious test case and corresponding fix
Skip Montanaro <skip@pobox.com>
parents: 33488
diff changeset
702 if i > j:
22494
7224a1df4809 add InvalidURL exception - raised if port is given but empty or non-numeric
Skip Montanaro <skip@pobox.com>
parents: 22403
diff changeset
703 try:
7224a1df4809 add InvalidURL exception - raised if port is given but empty or non-numeric
Skip Montanaro <skip@pobox.com>
parents: 22403
diff changeset
704 port = int(host[i+1:])
7224a1df4809 add InvalidURL exception - raised if port is given but empty or non-numeric
Skip Montanaro <skip@pobox.com>
parents: 22403
diff changeset
705 except ValueError:
24088
b74d4a59793a Convert raise to call exception class. Add whitespace.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24042
diff changeset
706 raise InvalidURL("nonnumeric port: '%s'" % host[i+1:])
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
707 host = host[:i]
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
708 else:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
709 port = self.default_port
33761
d1228127d294 SF bug #1046855: httplib index out of range
Raymond Hettinger <python@rcn.com>
parents: 33506
diff changeset
710 if host and host[0] == '[' and host[-1] == ']':
33496
9e2b94a3b5dc Remove tabs.
Brett Cannon <bcannon@gmail.com>
parents: 33493
diff changeset
711 host = host[1:-1]
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
712 self.host = host
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
713 self.port = port
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
714
14672
1b20ad6eb501 Do not close socket when a Content-Length is 0. This make the
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14560
diff changeset
715 def set_debuglevel(self, level):
1b20ad6eb501 Do not close socket when a Content-Length is 0. This make the
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14560
diff changeset
716 self.debuglevel = level
1b20ad6eb501 Do not close socket when a Content-Length is 0. This make the
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14560
diff changeset
717
54298
e7b4a443f764 Fixed Issue1424152, urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 51951
diff changeset
718 def _tunnel(self):
e7b4a443f764 Fixed Issue1424152, urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 51951
diff changeset
719 self._set_hostport(self._tunnel_host, self._tunnel_port)
57799
10970f64ba33 Fix for issue 7291 - urllib2 cannot handle https with proxy requiring auth
Senthil Kumaran <orsenthil@gmail.com>
parents: 56528
diff changeset
720 self.send("CONNECT %s:%d HTTP/1.0\r\n" % (self.host, self.port))
10970f64ba33 Fix for issue 7291 - urllib2 cannot handle https with proxy requiring auth
Senthil Kumaran <orsenthil@gmail.com>
parents: 56528
diff changeset
721 for header, value in self._tunnel_headers.iteritems():
10970f64ba33 Fix for issue 7291 - urllib2 cannot handle https with proxy requiring auth
Senthil Kumaran <orsenthil@gmail.com>
parents: 56528
diff changeset
722 self.send("%s: %s\r\n" % (header, value))
10970f64ba33 Fix for issue 7291 - urllib2 cannot handle https with proxy requiring auth
Senthil Kumaran <orsenthil@gmail.com>
parents: 56528
diff changeset
723 self.send("\r\n")
54298
e7b4a443f764 Fixed Issue1424152, urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 51951
diff changeset
724 response = self.response_class(self.sock, strict = self.strict,
e7b4a443f764 Fixed Issue1424152, urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 51951
diff changeset
725 method = self._method)
e7b4a443f764 Fixed Issue1424152, urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 51951
diff changeset
726 (version, code, message) = response._read_status()
e7b4a443f764 Fixed Issue1424152, urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 51951
diff changeset
727
e7b4a443f764 Fixed Issue1424152, urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 51951
diff changeset
728 if code != 200:
e7b4a443f764 Fixed Issue1424152, urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 51951
diff changeset
729 self.close()
56528
64e19a689886 Fix py3k warnings in httplib
Antoine Pitrou <solipsis@pitrou.net>
parents: 56285
diff changeset
730 raise socket.error("Tunnel connection failed: %d %s" % (code,
64e19a689886 Fix py3k warnings in httplib
Antoine Pitrou <solipsis@pitrou.net>
parents: 56285
diff changeset
731 message.strip()))
54298
e7b4a443f764 Fixed Issue1424152, urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 51951
diff changeset
732 while True:
e7b4a443f764 Fixed Issue1424152, urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 51951
diff changeset
733 line = response.fp.readline()
e7b4a443f764 Fixed Issue1424152, urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 51951
diff changeset
734 if line == '\r\n': break
e7b4a443f764 Fixed Issue1424152, urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 51951
diff changeset
735
e7b4a443f764 Fixed Issue1424152, urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 51951
diff changeset
736
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
737 def connect(self):
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
738 """Connect to the host and port specified in __init__."""
41305
8be744d5c567 Some nits.
Georg Brandl <georg@python.org>
parents: 41288
diff changeset
739 self.sock = socket.create_connection((self.host,self.port),
58113
a19ad299095f issue3972: HTTPConnection and HTTPSConnection now support a
Gregory P. Smith <greg@mad-scientist.com>
parents: 57799
diff changeset
740 self.timeout, self.source_address)
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
741
54298
e7b4a443f764 Fixed Issue1424152, urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 51951
diff changeset
742 if self._tunnel_host:
e7b4a443f764 Fixed Issue1424152, urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 51951
diff changeset
743 self._tunnel()
e7b4a443f764 Fixed Issue1424152, urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 51951
diff changeset
744
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
745 def close(self):
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
746 """Close the connection to the HTTP server."""
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
747 if self.sock:
15868
3060cbfa01be Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15859
diff changeset
748 self.sock.close() # close it manually... there may be other refs
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
749 self.sock = None
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
750 if self.__response:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
751 self.__response.close()
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
752 self.__response = None
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
753 self.__state = _CS_IDLE
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
754
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
755 def send(self, str):
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
756 """Send `str' to the server."""
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
757 if self.sock is None:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
758 if self.auto_open:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
759 self.connect()
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
760 else:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
761 raise NotConnected()
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
762
14672
1b20ad6eb501 Do not close socket when a Content-Length is 0. This make the
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14560
diff changeset
763 if self.debuglevel > 0:
1b20ad6eb501 Do not close socket when a Content-Length is 0. This make the
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14560
diff changeset
764 print "send:", repr(str)
63194
8072dfd97e1d Forward port r70643 (#5542) + part of r83120 (just remove the comment)
Victor Stinner <victor.stinner@haypocalc.com>
parents: 62064
diff changeset
765 blocksize = 8192
8072dfd97e1d Forward port r70643 (#5542) + part of r83120 (just remove the comment)
Victor Stinner <victor.stinner@haypocalc.com>
parents: 62064
diff changeset
766 if hasattr(str,'read') and not isinstance(str, array):
8072dfd97e1d Forward port r70643 (#5542) + part of r83120 (just remove the comment)
Victor Stinner <victor.stinner@haypocalc.com>
parents: 62064
diff changeset
767 if self.debuglevel > 0: print "sendIng a read()able"
8072dfd97e1d Forward port r70643 (#5542) + part of r83120 (just remove the comment)
Victor Stinner <victor.stinner@haypocalc.com>
parents: 62064
diff changeset
768 data = str.read(blocksize)
8072dfd97e1d Forward port r70643 (#5542) + part of r83120 (just remove the comment)
Victor Stinner <victor.stinner@haypocalc.com>
parents: 62064
diff changeset
769 while data:
8072dfd97e1d Forward port r70643 (#5542) + part of r83120 (just remove the comment)
Victor Stinner <victor.stinner@haypocalc.com>
parents: 62064
diff changeset
770 self.sock.sendall(data)
8072dfd97e1d Forward port r70643 (#5542) + part of r83120 (just remove the comment)
Victor Stinner <victor.stinner@haypocalc.com>
parents: 62064
diff changeset
771 data = str.read(blocksize)
8072dfd97e1d Forward port r70643 (#5542) + part of r83120 (just remove the comment)
Victor Stinner <victor.stinner@haypocalc.com>
parents: 62064
diff changeset
772 else:
8072dfd97e1d Forward port r70643 (#5542) + part of r83120 (just remove the comment)
Victor Stinner <victor.stinner@haypocalc.com>
parents: 62064
diff changeset
773 self.sock.sendall(str)
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
774
24262
3b571e32a6e4 Send HTTP requests with a single send() call instead of many.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24216
diff changeset
775 def _output(self, s):
3b571e32a6e4 Send HTTP requests with a single send() call instead of many.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24216
diff changeset
776 """Add a line of output to the current request buffer.
24674
3269bdd65988 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 24378
diff changeset
777
24264
18c1b290e484 Fix typos and such caught by the pycheckerbot.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24263
diff changeset
778 Assumes that the line does *not* end with \\r\\n.
24262
3b571e32a6e4 Send HTTP requests with a single send() call instead of many.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24216
diff changeset
779 """
3b571e32a6e4 Send HTTP requests with a single send() call instead of many.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24216
diff changeset
780 self._buffer.append(s)
3b571e32a6e4 Send HTTP requests with a single send() call instead of many.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24216
diff changeset
781
49890
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
782 def _send_output(self, message_body=None):
24262
3b571e32a6e4 Send HTTP requests with a single send() call instead of many.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24216
diff changeset
783 """Send the currently buffered request and clear the buffer.
3b571e32a6e4 Send HTTP requests with a single send() call instead of many.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24216
diff changeset
784
24264
18c1b290e484 Fix typos and such caught by the pycheckerbot.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24263
diff changeset
785 Appends an extra \\r\\n to the buffer.
50672
07bbc2e20aa4 Issue 4336: HTTPRequest._send_output() now deals with the case of the message body not being a string. This allows clients to use endheaders(message_body) instead of endheaders() + send(message_body) without making any extra checks.
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 49890
diff changeset
786 A message_body may be specified, to be appended to the request.
24262
3b571e32a6e4 Send HTTP requests with a single send() call instead of many.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24216
diff changeset
787 """
3b571e32a6e4 Send HTTP requests with a single send() call instead of many.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24216
diff changeset
788 self._buffer.extend(("", ""))
3b571e32a6e4 Send HTTP requests with a single send() call instead of many.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24216
diff changeset
789 msg = "\r\n".join(self._buffer)
3b571e32a6e4 Send HTTP requests with a single send() call instead of many.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24216
diff changeset
790 del self._buffer[:]
49890
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
791 # If msg and message_body are sent in a single send() call,
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
792 # it will avoid performance problems caused by the interaction
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
793 # between delayed ack and the Nagle algorithim.
50672
07bbc2e20aa4 Issue 4336: HTTPRequest._send_output() now deals with the case of the message body not being a string. This allows clients to use endheaders(message_body) instead of endheaders() + send(message_body) without making any extra checks.
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 49890
diff changeset
794 if isinstance(message_body, str):
49890
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
795 msg += message_body
50672
07bbc2e20aa4 Issue 4336: HTTPRequest._send_output() now deals with the case of the message body not being a string. This allows clients to use endheaders(message_body) instead of endheaders() + send(message_body) without making any extra checks.
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 49890
diff changeset
796 message_body = None
24262
3b571e32a6e4 Send HTTP requests with a single send() call instead of many.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24216
diff changeset
797 self.send(msg)
50672
07bbc2e20aa4 Issue 4336: HTTPRequest._send_output() now deals with the case of the message body not being a string. This allows clients to use endheaders(message_body) instead of endheaders() + send(message_body) without making any extra checks.
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 49890
diff changeset
798 if message_body is not None:
07bbc2e20aa4 Issue 4336: HTTPRequest._send_output() now deals with the case of the message body not being a string. This allows clients to use endheaders(message_body) instead of endheaders() + send(message_body) without making any extra checks.
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 49890
diff changeset
799 #message_body was not a string (i.e. it is a file) and
07bbc2e20aa4 Issue 4336: HTTPRequest._send_output() now deals with the case of the message body not being a string. This allows clients to use endheaders(message_body) instead of endheaders() + send(message_body) without making any extra checks.
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 49890
diff changeset
800 #we must run the risk of Nagle
07bbc2e20aa4 Issue 4336: HTTPRequest._send_output() now deals with the case of the message body not being a string. This allows clients to use endheaders(message_body) instead of endheaders() + send(message_body) without making any extra checks.
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 49890
diff changeset
801 self.send(message_body)
24262
3b571e32a6e4 Send HTTP requests with a single send() call instead of many.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24216
diff changeset
802
30976
ead4c61219d0 Patch #831747: Add skip_accept_encoding parameter to putrequest.
Martin v. Löwis <martin@v.loewis.de>
parents: 30888
diff changeset
803 def putrequest(self, method, url, skip_host=0, skip_accept_encoding=0):
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
804 """Send a request to the server.
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
805
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
806 `method' specifies an HTTP request method, e.g. 'GET'.
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
807 `url' specifies the object being requested, e.g. '/index.html'.
30976
ead4c61219d0 Patch #831747: Add skip_accept_encoding parameter to putrequest.
Martin v. Löwis <martin@v.loewis.de>
parents: 30888
diff changeset
808 `skip_host' if True does not add automatically a 'Host:' header
ead4c61219d0 Patch #831747: Add skip_accept_encoding parameter to putrequest.
Martin v. Löwis <martin@v.loewis.de>
parents: 30888
diff changeset
809 `skip_accept_encoding' if True does not add automatically an
ead4c61219d0 Patch #831747: Add skip_accept_encoding parameter to putrequest.
Martin v. Löwis <martin@v.loewis.de>
parents: 30888
diff changeset
810 'Accept-Encoding:' header
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
811 """
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
812
29584
da0248fcfc52 Deal with a couple XXX comments which asked questions.
Greg Stein <gstein@lyra.org>
parents: 29465
diff changeset
813 # if a prior response has been completed, then forget about it.
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
814 if self.__response and self.__response.isclosed():
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
815 self.__response = None
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
816
31389
aca1ce1b4b61 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 30976
diff changeset
817
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
818 # in certain cases, we cannot issue another request on this connection.
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
819 # this occurs when:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
820 # 1) we are in the process of sending a request. (_CS_REQ_STARTED)
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
821 # 2) a response to a previous request has signalled that it is going
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
822 # to close the connection upon completion.
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
823 # 3) the headers for the previous response have not been read, thus
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
824 # we cannot determine whether point (2) is true. (_CS_REQ_SENT)
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
825 #
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
826 # if there is no prior response, then we can request at will.
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
827 #
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
828 # if point (2) is true, then we will have passed the socket to the
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
829 # response (effectively meaning, "there is no prior response"), and
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
830 # will open a new one when a new request is made.
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
831 #
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
832 # Note: if a prior response exists, then we *can* start a new request.
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
833 # We are not allowed to begin fetching the response to this new
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
834 # request, however, until that prior response is complete.
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
835 #
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
836 if self.__state == _CS_IDLE:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
837 self.__state = _CS_REQ_STARTED
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
838 else:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
839 raise CannotSendRequest()
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
840
28978
83fe0f5c9359 SF bug 622042: Don't expect response body from HEAD request.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 28671
diff changeset
841 # Save the method we use, we need it later in the response phase
83fe0f5c9359 SF bug 622042: Don't expect response body from HEAD request.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 28671
diff changeset
842 self._method = method
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
843 if not url:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
844 url = '/'
24262
3b571e32a6e4 Send HTTP requests with a single send() call instead of many.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24216
diff changeset
845 str = '%s %s %s' % (method, url, self._http_vsn_str)
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
846
24262
3b571e32a6e4 Send HTTP requests with a single send() call instead of many.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24216
diff changeset
847 self._output(str)
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
848
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
849 if self._http_vsn == 11:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
850 # Issue some standard headers for better HTTP/1.1 compliance
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
851
22202
5f2c0f6f0af7 Fix SF bug 525520.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 22192
diff changeset
852 if not skip_host:
5f2c0f6f0af7 Fix SF bug 525520.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 22192
diff changeset
853 # this header is issued *only* for HTTP/1.1
5f2c0f6f0af7 Fix SF bug 525520.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 22192
diff changeset
854 # connections. more specifically, this means it is
5f2c0f6f0af7 Fix SF bug 525520.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 22192
diff changeset
855 # only issued when the client uses the new
5f2c0f6f0af7 Fix SF bug 525520.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 22192
diff changeset
856 # HTTPConnection() class. backwards-compat clients
5f2c0f6f0af7 Fix SF bug 525520.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 22192
diff changeset
857 # will be using HTTP/1.0 and those clients may be
5f2c0f6f0af7 Fix SF bug 525520.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 22192
diff changeset
858 # issuing this header themselves. we should NOT issue
5f2c0f6f0af7 Fix SF bug 525520.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 22192
diff changeset
859 # it twice; some web servers (such as Apache) barf
5f2c0f6f0af7 Fix SF bug 525520.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 22192
diff changeset
860 # when they see two Host: headers
15859
31967a233600 SF Patch #103232 by dougfort: Preserve Nonstandard Port Number in Host
Guido van Rossum <guido@python.org>
parents: 15845
diff changeset
861
22202
5f2c0f6f0af7 Fix SF bug 525520.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 22192
diff changeset
862 # If we need a non-standard port,include it in the
5f2c0f6f0af7 Fix SF bug 525520.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 22192
diff changeset
863 # header. If the request is going through a proxy,
5f2c0f6f0af7 Fix SF bug 525520.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 22192
diff changeset
864 # but the host of the actual URL, not the host of the
5f2c0f6f0af7 Fix SF bug 525520.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 22192
diff changeset
865 # proxy.
22192
044f9e71320f SF bug report #405939: wrong Host header with proxy
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 21962
diff changeset
866
22202
5f2c0f6f0af7 Fix SF bug 525520.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 22192
diff changeset
867 netloc = ''
5f2c0f6f0af7 Fix SF bug 525520.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 22192
diff changeset
868 if url.startswith('http'):
5f2c0f6f0af7 Fix SF bug 525520.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 22192
diff changeset
869 nil, netloc, nil, nil, nil = urlsplit(url)
5f2c0f6f0af7 Fix SF bug 525520.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 22192
diff changeset
870
5f2c0f6f0af7 Fix SF bug 525520.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 22192
diff changeset
871 if netloc:
38104
88b89f96b97f RFE #1472176: In httplib, don't encode the netloc and hostname with "idna" if not necessary.
Georg Brandl <georg@python.org>
parents: 36559
diff changeset
872 try:
88b89f96b97f RFE #1472176: In httplib, don't encode the netloc and hostname with "idna" if not necessary.
Georg Brandl <georg@python.org>
parents: 36559
diff changeset
873 netloc_enc = netloc.encode("ascii")
88b89f96b97f RFE #1472176: In httplib, don't encode the netloc and hostname with "idna" if not necessary.
Georg Brandl <georg@python.org>
parents: 36559
diff changeset
874 except UnicodeEncodeError:
88b89f96b97f RFE #1472176: In httplib, don't encode the netloc and hostname with "idna" if not necessary.
Georg Brandl <georg@python.org>
parents: 36559
diff changeset
875 netloc_enc = netloc.encode("idna")
88b89f96b97f RFE #1472176: In httplib, don't encode the netloc and hostname with "idna" if not necessary.
Georg Brandl <georg@python.org>
parents: 36559
diff changeset
876 self.putheader('Host', netloc_enc)
22202
5f2c0f6f0af7 Fix SF bug 525520.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 22192
diff changeset
877 else:
38104
88b89f96b97f RFE #1472176: In httplib, don't encode the netloc and hostname with "idna" if not necessary.
Georg Brandl <georg@python.org>
parents: 36559
diff changeset
878 try:
88b89f96b97f RFE #1472176: In httplib, don't encode the netloc and hostname with "idna" if not necessary.
Georg Brandl <georg@python.org>
parents: 36559
diff changeset
879 host_enc = self.host.encode("ascii")
88b89f96b97f RFE #1472176: In httplib, don't encode the netloc and hostname with "idna" if not necessary.
Georg Brandl <georg@python.org>
parents: 36559
diff changeset
880 except UnicodeEncodeError:
88b89f96b97f RFE #1472176: In httplib, don't encode the netloc and hostname with "idna" if not necessary.
Georg Brandl <georg@python.org>
parents: 36559
diff changeset
881 host_enc = self.host.encode("idna")
47909
08c5c8c114b2 - Issue #3094: httplib.HTTPSConnection Host: headers no longer include the
Gregory P. Smith <greg@mad-scientist.com>
parents: 47372
diff changeset
882 if self.port == self.default_port:
38104
88b89f96b97f RFE #1472176: In httplib, don't encode the netloc and hostname with "idna" if not necessary.
Georg Brandl <georg@python.org>
parents: 36559
diff changeset
883 self.putheader('Host', host_enc)
88b89f96b97f RFE #1472176: In httplib, don't encode the netloc and hostname with "idna" if not necessary.
Georg Brandl <georg@python.org>
parents: 36559
diff changeset
884 else:
88b89f96b97f RFE #1472176: In httplib, don't encode the netloc and hostname with "idna" if not necessary.
Georg Brandl <georg@python.org>
parents: 36559
diff changeset
885 self.putheader('Host', "%s:%s" % (host_enc, self.port))
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
886
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
887 # note: we are assuming that clients will not attempt to set these
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
888 # headers since *this* library must deal with the
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
889 # consequences. this also means that when the supporting
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
890 # libraries are updated to recognize other forms, then this
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
891 # code should be changed (removed or updated).
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
892
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
893 # we only want a Content-Encoding of "identity" since we don't
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
894 # support encodings such as x-gzip or x-deflate.
30976
ead4c61219d0 Patch #831747: Add skip_accept_encoding parameter to putrequest.
Martin v. Löwis <martin@v.loewis.de>
parents: 30888
diff changeset
895 if not skip_accept_encoding:
ead4c61219d0 Patch #831747: Add skip_accept_encoding parameter to putrequest.
Martin v. Löwis <martin@v.loewis.de>
parents: 30888
diff changeset
896 self.putheader('Accept-Encoding', 'identity')
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
897
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
898 # we can accept "chunked" Transfer-Encodings, but no others
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
899 # NOTE: no TE header implies *only* "chunked"
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
900 #self.putheader('TE', 'chunked')
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
901
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
902 # if TE is supplied in the header, then it must appear in a
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
903 # Connection header.
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
904 #self.putheader('Connection', 'TE')
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
905
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
906 else:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
907 # For HTTP/1.0, the server will assume "not chunked"
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
908 pass
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
909
49726
0ad07078433a The docs for httplib.HTTPConnection.putheader() have claimed for quite a while
Brett Cannon <bcannon@gmail.com>
parents: 48948
diff changeset
910 def putheader(self, header, *values):
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
911 """Send a request header line to the server.
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
912
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
913 For example: h.putheader('Accept', 'text/html')
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
914 """
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
915 if self.__state != _CS_REQ_STARTED:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
916 raise CannotSendHeader()
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
917
49726
0ad07078433a The docs for httplib.HTTPConnection.putheader() have claimed for quite a while
Brett Cannon <bcannon@gmail.com>
parents: 48948
diff changeset
918 str = '%s: %s' % (header, '\r\n\t'.join(values))
24262
3b571e32a6e4 Send HTTP requests with a single send() call instead of many.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24216
diff changeset
919 self._output(str)
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
920
49890
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
921 def endheaders(self, message_body=None):
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
922 """Indicate that the last header line has been sent to the server.
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
923
49890
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
924 This method sends the request to the server. The optional
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
925 message_body argument can be used to pass message body
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
926 associated with the request. The message body will be sent in
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
927 the same packet as the message headers if possible. The
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
928 message_body should be a string.
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
929 """
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
930 if self.__state == _CS_REQ_STARTED:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
931 self.__state = _CS_REQ_SENT
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
932 else:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
933 raise CannotSendHeader()
49890
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
934 self._send_output(message_body)
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
935
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
936 def request(self, method, url, body=None, headers={}):
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
937 """Send a complete request to the server."""
63194
8072dfd97e1d Forward port r70643 (#5542) + part of r83120 (just remove the comment)
Victor Stinner <victor.stinner@haypocalc.com>
parents: 62064
diff changeset
938 self._send_request(method, url, body, headers)
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
939
49890
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
940 def _set_content_length(self, body):
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
941 # Set the content-length based on the body.
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
942 thelen = None
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
943 try:
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
944 thelen = str(len(body))
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
945 except TypeError, te:
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
946 # If this is a file-like object, try to
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
947 # fstat its file descriptor
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
948 try:
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
949 thelen = str(os.fstat(body.fileno()).st_size)
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
950 except (AttributeError, OSError):
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
951 # Don't send a length if this failed
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
952 if self.debuglevel > 0: print "Cannot stat!!"
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
953
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
954 if thelen is not None:
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
955 self.putheader('Content-Length', thelen)
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
956
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
957 def _send_request(self, method, url, body, headers):
63194
8072dfd97e1d Forward port r70643 (#5542) + part of r83120 (just remove the comment)
Victor Stinner <victor.stinner@haypocalc.com>
parents: 62064
diff changeset
958 # Honor explicitly requested Host: and Accept-Encoding: headers.
33001
4fd59651fe3f SF bug 874842 and patch 997626: httplib bugs
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 32252
diff changeset
959 header_names = dict.fromkeys([k.lower() for k in headers])
4fd59651fe3f SF bug 874842 and patch 997626: httplib bugs
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 32252
diff changeset
960 skips = {}
4fd59651fe3f SF bug 874842 and patch 997626: httplib bugs
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 32252
diff changeset
961 if 'host' in header_names:
4fd59651fe3f SF bug 874842 and patch 997626: httplib bugs
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 32252
diff changeset
962 skips['skip_host'] = 1
4fd59651fe3f SF bug 874842 and patch 997626: httplib bugs
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 32252
diff changeset
963 if 'accept-encoding' in header_names:
4fd59651fe3f SF bug 874842 and patch 997626: httplib bugs
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 32252
diff changeset
964 skips['skip_accept_encoding'] = 1
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
965
33001
4fd59651fe3f SF bug 874842 and patch 997626: httplib bugs
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 32252
diff changeset
966 self.putrequest(method, url, **skips)
4fd59651fe3f SF bug 874842 and patch 997626: httplib bugs
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 32252
diff changeset
967
4fd59651fe3f SF bug 874842 and patch 997626: httplib bugs
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 32252
diff changeset
968 if body and ('content-length' not in header_names):
49890
ef0c5ca45441 Send HTTP headers and message body in a single send() call.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 49726
diff changeset
969 self._set_content_length(body)
28083
29965d0d4650 Module review:
Raymond Hettinger <python@rcn.com>
parents: 28063
diff changeset
970 for hdr, value in headers.iteritems():
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
971 self.putheader(hdr, value)
50672
07bbc2e20aa4 Issue 4336: HTTPRequest._send_output() now deals with the case of the message body not being a string. This allows clients to use endheaders(message_body) instead of endheaders() + send(message_body) without making any extra checks.
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 49890
diff changeset
972 self.endheaders(body)
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
973
50734
d09d6fe31b61 Issue 4879: Allow buffering for HTTPResponse
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 50672
diff changeset
974 def getresponse(self, buffering=False):
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
975 "Get the response from the server."
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
976
29584
da0248fcfc52 Deal with a couple XXX comments which asked questions.
Greg Stein <gstein@lyra.org>
parents: 29465
diff changeset
977 # if a prior response has been completed, then forget about it.
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
978 if self.__response and self.__response.isclosed():
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
979 self.__response = None
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
980
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
981 #
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
982 # if a prior response exists, then it must be completed (otherwise, we
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
983 # cannot read this response's header to determine the connection-close
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
984 # behavior)
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
985 #
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
986 # note: if a prior response existed, but was connection-close, then the
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
987 # socket and response were made independent of this HTTPConnection
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
988 # object since a new request requires that we open a whole new
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
989 # connection
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
990 #
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
991 # this means the prior response had one of two states:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
992 # 1) will_close: this connection was reset and the prior socket and
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
993 # response operate independently
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
994 # 2) persistent: the response was retained and we await its
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
995 # isclosed() status to become true.
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
996 #
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
997 if self.__state != _CS_REQ_SENT or self.__response:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
998 raise ResponseNotReady()
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
999
50734
d09d6fe31b61 Issue 4879: Allow buffering for HTTPResponse
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 50672
diff changeset
1000 args = (self.sock,)
d09d6fe31b61 Issue 4879: Allow buffering for HTTPResponse
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 50672
diff changeset
1001 kwds = {"strict":self.strict, "method":self._method}
14672
1b20ad6eb501 Do not close socket when a Content-Length is 0. This make the
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14560
diff changeset
1002 if self.debuglevel > 0:
50734
d09d6fe31b61 Issue 4879: Allow buffering for HTTPResponse
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 50672
diff changeset
1003 args += (self.debuglevel,)
d09d6fe31b61 Issue 4879: Allow buffering for HTTPResponse
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 50672
diff changeset
1004 if buffering:
d09d6fe31b61 Issue 4879: Allow buffering for HTTPResponse
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 50672
diff changeset
1005 #only add this keyword if non-default, for compatibility with
d09d6fe31b61 Issue 4879: Allow buffering for HTTPResponse
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 50672
diff changeset
1006 #other response_classes.
d09d6fe31b61 Issue 4879: Allow buffering for HTTPResponse
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 50672
diff changeset
1007 kwds["buffering"] = True;
d09d6fe31b61 Issue 4879: Allow buffering for HTTPResponse
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 50672
diff changeset
1008 response = self.response_class(*args, **kwds)
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1009
24216
c8a19a428358 Change _begin() back to begin().
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24168
diff changeset
1010 response.begin()
24038
9377158652dd Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23620
diff changeset
1011 assert response.will_close != _UNKNOWN
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1012 self.__state = _CS_IDLE
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1013
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1014 if response.will_close:
41285
02574f485b15 Bug #978833: Revert r50844, as it broke _socketobject.dup.
Martin v. Löwis <martin@v.loewis.de>
parents: 40732
diff changeset
1015 # this effectively passes the connection to the response
02574f485b15 Bug #978833: Revert r50844, as it broke _socketobject.dup.
Martin v. Löwis <martin@v.loewis.de>
parents: 40732
diff changeset
1016 self.close()
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1017 else:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1018 # remember this, so we can tell when it is complete
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1019 self.__response = response
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1020
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1021 return response
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1022
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1023
13787
1f966f0d467c add support for HTTPS
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 13636
diff changeset
1024 class HTTP:
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1025 "Compatibility class with httplib.py from 1.5."
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1026
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1027 _http_vsn = 10
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1028 _http_vsn_str = 'HTTP/1.0'
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1029
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1030 debuglevel = 0
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1031
13787
1f966f0d467c add support for HTTPS
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 13636
diff changeset
1032 _connection_class = HTTPConnection
1f966f0d467c add support for HTTPS
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 13636
diff changeset
1033
24119
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1034 def __init__(self, host='', port=None, strict=None):
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1035 "Provide a default host, since the superclass requires one."
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1036
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1037 # some joker passed 0 explicitly, meaning default port
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1038 if port == 0:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1039 port = None
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1040
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1041 # Note that we may pass an empty string as the host; this will throw
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1042 # an error when we attempt to connect. Presumably, the client code
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1043 # will call connect before then, with a proper host.
24119
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1044 self._setup(self._connection_class(host, port, strict))
19367
17802077e77c Resolve patch #449367.
Greg Stein <gstein@lyra.org>
parents: 18902
diff changeset
1045
17802077e77c Resolve patch #449367.
Greg Stein <gstein@lyra.org>
parents: 18902
diff changeset
1046 def _setup(self, conn):
17802077e77c Resolve patch #449367.
Greg Stein <gstein@lyra.org>
parents: 18902
diff changeset
1047 self._conn = conn
17802077e77c Resolve patch #449367.
Greg Stein <gstein@lyra.org>
parents: 18902
diff changeset
1048
13787
1f966f0d467c add support for HTTPS
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 13636
diff changeset
1049 # set up delegation to flesh out interface
19367
17802077e77c Resolve patch #449367.
Greg Stein <gstein@lyra.org>
parents: 18902
diff changeset
1050 self.send = conn.send
17802077e77c Resolve patch #449367.
Greg Stein <gstein@lyra.org>
parents: 18902
diff changeset
1051 self.putrequest = conn.putrequest
49726
0ad07078433a The docs for httplib.HTTPConnection.putheader() have claimed for quite a while
Brett Cannon <bcannon@gmail.com>
parents: 48948
diff changeset
1052 self.putheader = conn.putheader
19367
17802077e77c Resolve patch #449367.
Greg Stein <gstein@lyra.org>
parents: 18902
diff changeset
1053 self.endheaders = conn.endheaders
17802077e77c Resolve patch #449367.
Greg Stein <gstein@lyra.org>
parents: 18902
diff changeset
1054 self.set_debuglevel = conn.set_debuglevel
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1055
19367
17802077e77c Resolve patch #449367.
Greg Stein <gstein@lyra.org>
parents: 18902
diff changeset
1056 conn._http_vsn = self._http_vsn
17802077e77c Resolve patch #449367.
Greg Stein <gstein@lyra.org>
parents: 18902
diff changeset
1057 conn._http_vsn_str = self._http_vsn_str
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1058
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1059 self.file = None
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1060
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1061 def connect(self, host=None, port=None):
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1062 "Accept arguments to set the host/port, since the superclass doesn't."
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1063
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1064 if host is not None:
13787
1f966f0d467c add support for HTTPS
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 13636
diff changeset
1065 self._conn._set_hostport(host, port)
1f966f0d467c add support for HTTPS
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 13636
diff changeset
1066 self._conn.connect()
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1067
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1068 def getfile(self):
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1069 "Provide a getfile, since the superclass' does not use this concept."
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1070 return self.file
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1071
50734
d09d6fe31b61 Issue 4879: Allow buffering for HTTPResponse
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 50672
diff changeset
1072 def getreply(self, buffering=False):
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1073 """Compat definition since superclass does not define it.
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1074
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1075 Returns a tuple consisting of:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1076 - server status code (e.g. '200' if all goes well)
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1077 - server "reason" corresponding to status code
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1078 - any RFC822 headers in the response from the server
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1079 """
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1080 try:
50734
d09d6fe31b61 Issue 4879: Allow buffering for HTTPResponse
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 50672
diff changeset
1081 if not buffering:
d09d6fe31b61 Issue 4879: Allow buffering for HTTPResponse
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 50672
diff changeset
1082 response = self._conn.getresponse()
d09d6fe31b61 Issue 4879: Allow buffering for HTTPResponse
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 50672
diff changeset
1083 else:
d09d6fe31b61 Issue 4879: Allow buffering for HTTPResponse
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 50672
diff changeset
1084 #only add this keyword if non-default for compatibility
d09d6fe31b61 Issue 4879: Allow buffering for HTTPResponse
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 50672
diff changeset
1085 #with other connection classes
d09d6fe31b61 Issue 4879: Allow buffering for HTTPResponse
Kristján Valur Jónsson <kristjan@ccpgames.com>
parents: 50672
diff changeset
1086 response = self._conn.getresponse(buffering)
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1087 except BadStatusLine, e:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1088 ### hmm. if getresponse() ever closes the socket on a bad request,
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1089 ### then we are going to have problems with self.sock
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1090
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1091 ### should we keep this behavior? do people use it?
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1092 # keep the socket open (as a file), and return it
13787
1f966f0d467c add support for HTTPS
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 13636
diff changeset
1093 self.file = self._conn.sock.makefile('rb', 0)
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1094
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1095 # close our socket -- we want to restart after any protocol error
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1096 self.close()
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1097
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1098 self.headers = None
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1099 return -1, e.line, None
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1100
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1101 self.headers = response.msg
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1102 self.file = response.fp
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1103 return response.status, response.reason, response.msg
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1104
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1105 def close(self):
13787
1f966f0d467c add support for HTTPS
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 13636
diff changeset
1106 self._conn.close()
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1107
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1108 # note that self.file == response.fp, which gets closed by the
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1109 # superclass. just clear the object ref here.
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1110 ### hmm. messy. if status==-1, then self.file is owned by us.
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1111 ### well... we aren't explicitly closing, but losing this ref will
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1112 ### do it
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1113 self.file = None
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1114
43008
4771639fc049 This contains a number of things:
Bill Janssen <janssen@parc.com>
parents: 42385
diff changeset
1115 try:
4771639fc049 This contains a number of things:
Bill Janssen <janssen@parc.com>
parents: 42385
diff changeset
1116 import ssl
4771639fc049 This contains a number of things:
Bill Janssen <janssen@parc.com>
parents: 42385
diff changeset
1117 except ImportError:
4771639fc049 This contains a number of things:
Bill Janssen <janssen@parc.com>
parents: 42385
diff changeset
1118 pass
4771639fc049 This contains a number of things:
Bill Janssen <janssen@parc.com>
parents: 42385
diff changeset
1119 else:
4771639fc049 This contains a number of things:
Bill Janssen <janssen@parc.com>
parents: 42385
diff changeset
1120 class HTTPSConnection(HTTPConnection):
4771639fc049 This contains a number of things:
Bill Janssen <janssen@parc.com>
parents: 42385
diff changeset
1121 "This class allows communication via SSL."
4771639fc049 This contains a number of things:
Bill Janssen <janssen@parc.com>
parents: 42385
diff changeset
1122
4771639fc049 This contains a number of things:
Bill Janssen <janssen@parc.com>
parents: 42385
diff changeset
1123 default_port = HTTPS_PORT
4771639fc049 This contains a number of things:
Bill Janssen <janssen@parc.com>
parents: 42385
diff changeset
1124
4771639fc049 This contains a number of things:
Bill Janssen <janssen@parc.com>
parents: 42385
diff changeset
1125 def __init__(self, host, port=None, key_file=None, cert_file=None,
58113
a19ad299095f issue3972: HTTPConnection and HTTPSConnection now support a
Gregory P. Smith <greg@mad-scientist.com>
parents: 57799
diff changeset
1126 strict=None, timeout=socket._GLOBAL_DEFAULT_TIMEOUT,
a19ad299095f issue3972: HTTPConnection and HTTPSConnection now support a
Gregory P. Smith <greg@mad-scientist.com>
parents: 57799
diff changeset
1127 source_address=None):
a19ad299095f issue3972: HTTPConnection and HTTPSConnection now support a
Gregory P. Smith <greg@mad-scientist.com>
parents: 57799
diff changeset
1128 HTTPConnection.__init__(self, host, port, strict, timeout,
a19ad299095f issue3972: HTTPConnection and HTTPSConnection now support a
Gregory P. Smith <greg@mad-scientist.com>
parents: 57799
diff changeset
1129 source_address)
43008
4771639fc049 This contains a number of things:
Bill Janssen <janssen@parc.com>
parents: 42385
diff changeset
1130 self.key_file = key_file
4771639fc049 This contains a number of things:
Bill Janssen <janssen@parc.com>
parents: 42385
diff changeset
1131 self.cert_file = cert_file
4771639fc049 This contains a number of things:
Bill Janssen <janssen@parc.com>
parents: 42385
diff changeset
1132
4771639fc049 This contains a number of things:
Bill Janssen <janssen@parc.com>
parents: 42385
diff changeset
1133 def connect(self):
4771639fc049 This contains a number of things:
Bill Janssen <janssen@parc.com>
parents: 42385
diff changeset
1134 "Connect to a host on a given (SSL) port."
4771639fc049 This contains a number of things:
Bill Janssen <janssen@parc.com>
parents: 42385
diff changeset
1135
58113
a19ad299095f issue3972: HTTPConnection and HTTPSConnection now support a
Gregory P. Smith <greg@mad-scientist.com>
parents: 57799
diff changeset
1136 sock = socket.create_connection((self.host, self.port),
a19ad299095f issue3972: HTTPConnection and HTTPSConnection now support a
Gregory P. Smith <greg@mad-scientist.com>
parents: 57799
diff changeset
1137 self.timeout, self.source_address)
54298
e7b4a443f764 Fixed Issue1424152, urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 51951
diff changeset
1138 if self._tunnel_host:
e7b4a443f764 Fixed Issue1424152, urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 51951
diff changeset
1139 self.sock = sock
e7b4a443f764 Fixed Issue1424152, urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 51951
diff changeset
1140 self._tunnel()
43325
193020bf52dc More work on SSL support.
Bill Janssen <janssen@parc.com>
parents: 43161
diff changeset
1141 self.sock = ssl.wrap_socket(sock, self.key_file, self.cert_file)
43008
4771639fc049 This contains a number of things:
Bill Janssen <janssen@parc.com>
parents: 42385
diff changeset
1142
43093
1d57fb7efb15 Don't lie in __all__ attributes when SSL is not available: only add the SSL
Thomas Wouters <thomas@python.org>
parents: 43008
diff changeset
1143 __all__.append("HTTPSConnection")
43008
4771639fc049 This contains a number of things:
Bill Janssen <janssen@parc.com>
parents: 42385
diff changeset
1144
13787
1f966f0d467c add support for HTTPS
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 13636
diff changeset
1145 class HTTPS(HTTP):
1f966f0d467c add support for HTTPS
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 13636
diff changeset
1146 """Compatibility with 1.5 httplib interface
1f966f0d467c add support for HTTPS
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 13636
diff changeset
1147
1f966f0d467c add support for HTTPS
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 13636
diff changeset
1148 Python 1.5.2 did not have an HTTPS class, but it defined an
1f966f0d467c add support for HTTPS
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 13636
diff changeset
1149 interface for sending http requests that is also useful for
15845
210bbe53b6d4 No text file relying on significant trailing whitespace is robust under
Tim Peters <tim.peters@gmail.com>
parents: 15844
diff changeset
1150 https.
13787
1f966f0d467c add support for HTTPS
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 13636
diff changeset
1151 """
1f966f0d467c add support for HTTPS
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 13636
diff changeset
1152
14740
8bb6dca66ecd Indent _connection_class so that it becomes HTTPS._connection_class.
Martin v. Löwis <martin@v.loewis.de>
parents: 14672
diff changeset
1153 _connection_class = HTTPSConnection
15845
210bbe53b6d4 No text file relying on significant trailing whitespace is robust under
Tim Peters <tim.peters@gmail.com>
parents: 15844
diff changeset
1154
24119
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1155 def __init__(self, host='', port=None, key_file=None, cert_file=None,
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1156 strict=None):
19367
17802077e77c Resolve patch #449367.
Greg Stein <gstein@lyra.org>
parents: 18902
diff changeset
1157 # provide a default host, pass the X509 cert info
17802077e77c Resolve patch #449367.
Greg Stein <gstein@lyra.org>
parents: 18902
diff changeset
1158
17802077e77c Resolve patch #449367.
Greg Stein <gstein@lyra.org>
parents: 18902
diff changeset
1159 # urf. compensate for bad input.
17802077e77c Resolve patch #449367.
Greg Stein <gstein@lyra.org>
parents: 18902
diff changeset
1160 if port == 0:
17802077e77c Resolve patch #449367.
Greg Stein <gstein@lyra.org>
parents: 18902
diff changeset
1161 port = None
24119
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1162 self._setup(self._connection_class(host, port, key_file,
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1163 cert_file, strict))
19367
17802077e77c Resolve patch #449367.
Greg Stein <gstein@lyra.org>
parents: 18902
diff changeset
1164
17802077e77c Resolve patch #449367.
Greg Stein <gstein@lyra.org>
parents: 18902
diff changeset
1165 # we never actually use these for anything, but we keep them
17802077e77c Resolve patch #449367.
Greg Stein <gstein@lyra.org>
parents: 18902
diff changeset
1166 # here for compatibility with post-1.5.2 CVS.
24119
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1167 self.key_file = key_file
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1168 self.cert_file = cert_file
19367
17802077e77c Resolve patch #449367.
Greg Stein <gstein@lyra.org>
parents: 18902
diff changeset
1169
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1170
43008
4771639fc049 This contains a number of things:
Bill Janssen <janssen@parc.com>
parents: 42385
diff changeset
1171 def FakeSocket (sock, sslobj):
43161
9cfad9a5c83b deprecate use of FakeSocket
Bill Janssen <janssen@parc.com>
parents: 43093
diff changeset
1172 warnings.warn("FakeSocket is deprecated, and won't be in 3.x. " +
43325
193020bf52dc More work on SSL support.
Bill Janssen <janssen@parc.com>
parents: 43161
diff changeset
1173 "Use the result of ssl.wrap_socket() directly instead.",
43161
9cfad9a5c83b deprecate use of FakeSocket
Bill Janssen <janssen@parc.com>
parents: 43093
diff changeset
1174 DeprecationWarning, stacklevel=2)
43008
4771639fc049 This contains a number of things:
Bill Janssen <janssen@parc.com>
parents: 42385
diff changeset
1175 return sslobj
4771639fc049 This contains a number of things:
Bill Janssen <janssen@parc.com>
parents: 42385
diff changeset
1176
4771639fc049 This contains a number of things:
Bill Janssen <janssen@parc.com>
parents: 42385
diff changeset
1177
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1178 class HTTPException(Exception):
24120
b51a578f5769 Fix SF bug #575360
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24119
diff changeset
1179 # Subclasses that define an __init__ must call Exception.__init__
b51a578f5769 Fix SF bug #575360
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24119
diff changeset
1180 # or define self.args. Otherwise, str() will fail.
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1181 pass
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1182
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1183 class NotConnected(HTTPException):
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1184 pass
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1185
22494
7224a1df4809 add InvalidURL exception - raised if port is given but empty or non-numeric
Skip Montanaro <skip@pobox.com>
parents: 22403
diff changeset
1186 class InvalidURL(HTTPException):
7224a1df4809 add InvalidURL exception - raised if port is given but empty or non-numeric
Skip Montanaro <skip@pobox.com>
parents: 22403
diff changeset
1187 pass
7224a1df4809 add InvalidURL exception - raised if port is given but empty or non-numeric
Skip Montanaro <skip@pobox.com>
parents: 22403
diff changeset
1188
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1189 class UnknownProtocol(HTTPException):
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1190 def __init__(self, version):
24120
b51a578f5769 Fix SF bug #575360
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24119
diff changeset
1191 self.args = version,
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1192 self.version = version
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1193
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1194 class UnknownTransferEncoding(HTTPException):
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1195 pass
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1196
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1197 class UnimplementedFileMode(HTTPException):
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1198 pass
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1199
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1200 class IncompleteRead(HTTPException):
51951
87115c727102 give httplib.IncompleteRead a more sane repr #4308
Benjamin Peterson <benjamin@python.org>
parents: 50734
diff changeset
1201 def __init__(self, partial, expected=None):
24120
b51a578f5769 Fix SF bug #575360
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24119
diff changeset
1202 self.args = partial,
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1203 self.partial = partial
51951
87115c727102 give httplib.IncompleteRead a more sane repr #4308
Benjamin Peterson <benjamin@python.org>
parents: 50734
diff changeset
1204 self.expected = expected
87115c727102 give httplib.IncompleteRead a more sane repr #4308
Benjamin Peterson <benjamin@python.org>
parents: 50734
diff changeset
1205 def __repr__(self):
87115c727102 give httplib.IncompleteRead a more sane repr #4308
Benjamin Peterson <benjamin@python.org>
parents: 50734
diff changeset
1206 if self.expected is not None:
87115c727102 give httplib.IncompleteRead a more sane repr #4308
Benjamin Peterson <benjamin@python.org>
parents: 50734
diff changeset
1207 e = ', %i more expected' % self.expected
87115c727102 give httplib.IncompleteRead a more sane repr #4308
Benjamin Peterson <benjamin@python.org>
parents: 50734
diff changeset
1208 else:
87115c727102 give httplib.IncompleteRead a more sane repr #4308
Benjamin Peterson <benjamin@python.org>
parents: 50734
diff changeset
1209 e = ''
87115c727102 give httplib.IncompleteRead a more sane repr #4308
Benjamin Peterson <benjamin@python.org>
parents: 50734
diff changeset
1210 return 'IncompleteRead(%i bytes read%s)' % (len(self.partial), e)
87115c727102 give httplib.IncompleteRead a more sane repr #4308
Benjamin Peterson <benjamin@python.org>
parents: 50734
diff changeset
1211 def __str__(self):
87115c727102 give httplib.IncompleteRead a more sane repr #4308
Benjamin Peterson <benjamin@python.org>
parents: 50734
diff changeset
1212 return repr(self)
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1213
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1214 class ImproperConnectionState(HTTPException):
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1215 pass
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1216
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1217 class CannotSendRequest(ImproperConnectionState):
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1218 pass
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1219
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1220 class CannotSendHeader(ImproperConnectionState):
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1221 pass
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1222
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1223 class ResponseNotReady(ImproperConnectionState):
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1224 pass
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1225
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1226 class BadStatusLine(HTTPException):
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1227 def __init__(self, line):
59107
45212a247ed4 Issue #7427: improve the representation of httplib.BadStatusLine exceptions.
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 58113
diff changeset
1228 if not line:
45212a247ed4 Issue #7427: improve the representation of httplib.BadStatusLine exceptions.
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 58113
diff changeset
1229 line = repr(line)
24120
b51a578f5769 Fix SF bug #575360
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24119
diff changeset
1230 self.args = line,
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1231 self.line = line
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1232
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1233 # for backwards compatibility
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1234 error = HTTPException
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1235
24119
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1236 class LineAndFileWrapper:
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1237 """A limited file-like object for HTTP/0.9 responses."""
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1238
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1239 # The status-line parsing code calls readline(), which normally
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1240 # get the HTTP status line. For a 0.9 response, however, this is
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1241 # actually the first line of the body! Clients need to get a
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1242 # readable file object that contains that line.
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1243
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1244 def __init__(self, line, file):
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1245 self._line = line
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1246 self._file = file
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1247 self._line_consumed = 0
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1248 self._line_offset = 0
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1249 self._line_left = len(line)
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1250
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1251 def __getattr__(self, attr):
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1252 return getattr(self._file, attr)
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1253
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1254 def _done(self):
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1255 # called when the last byte is read from the line. After the
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1256 # call, all read methods are delegated to the underlying file
28063
Skip Montanaro <skip@pobox.com>
parents: 26387
diff changeset
1257 # object.
24119
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1258 self._line_consumed = 1
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1259 self.read = self._file.read
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1260 self.readline = self._file.readline
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1261 self.readlines = self._file.readlines
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1262
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1263 def read(self, amt=None):
32252
a72bc4257336 Fix a bug that robotparser starves memory when the server responses
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31967
diff changeset
1264 if self._line_consumed:
a72bc4257336 Fix a bug that robotparser starves memory when the server responses
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31967
diff changeset
1265 return self._file.read(amt)
a72bc4257336 Fix a bug that robotparser starves memory when the server responses
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31967
diff changeset
1266 assert self._line_left
24119
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1267 if amt is None or amt > self._line_left:
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1268 s = self._line[self._line_offset:]
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1269 self._done()
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1270 if amt is None:
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1271 return s + self._file.read()
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1272 else:
24263
e6033254df21 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 24262
diff changeset
1273 return s + self._file.read(amt - len(s))
24119
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1274 else:
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1275 assert amt <= self._line_left
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1276 i = self._line_offset
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1277 j = i + amt
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1278 s = self._line[i:j]
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1279 self._line_offset = j
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1280 self._line_left -= amt
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1281 if self._line_left == 0:
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1282 self._done()
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1283 return s
24263
e6033254df21 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 24262
diff changeset
1284
24119
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1285 def readline(self):
32252
a72bc4257336 Fix a bug that robotparser starves memory when the server responses
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31967
diff changeset
1286 if self._line_consumed:
a72bc4257336 Fix a bug that robotparser starves memory when the server responses
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31967
diff changeset
1287 return self._file.readline()
a72bc4257336 Fix a bug that robotparser starves memory when the server responses
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31967
diff changeset
1288 assert self._line_left
24119
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1289 s = self._line[self._line_offset:]
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1290 self._done()
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1291 return s
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1292
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1293 def readlines(self, size=None):
32252
a72bc4257336 Fix a bug that robotparser starves memory when the server responses
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31967
diff changeset
1294 if self._line_consumed:
a72bc4257336 Fix a bug that robotparser starves memory when the server responses
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31967
diff changeset
1295 return self._file.readlines(size)
a72bc4257336 Fix a bug that robotparser starves memory when the server responses
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31967
diff changeset
1296 assert self._line_left
24119
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1297 L = [self._line[self._line_offset:]]
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1298 self._done()
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1299 if size is None:
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1300 return L + self._file.readlines()
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1301 else:
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1302 return L + self._file.readlines(size)
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1303
1555
6d35308261f4 Towards a standard access mechanism for URLs.
Guido van Rossum <guido@python.org>
parents:
diff changeset
1304 def test():
6787
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
1305 """Test this module.
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
1306
24168
fdd836575248 Fix for SF bug 579107.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24123
diff changeset
1307 A hodge podge of tests collected here, because they have too many
fdd836575248 Fix for SF bug 579107.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24123
diff changeset
1308 external dependencies for the regular test suite.
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1309 """
6787
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
1310
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
1311 import sys
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
1312 import getopt
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
1313 opts, args = getopt.getopt(sys.argv[1:], 'd')
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
1314 dl = 0
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
1315 for o, a in opts:
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
1316 if o == '-d': dl = dl + 1
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
1317 host = 'www.python.org'
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
1318 selector = '/'
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
1319 if args[0:]: host = args[0]
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
1320 if args[1:]: selector = args[1]
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
1321 h = HTTP()
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
1322 h.set_debuglevel(dl)
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
1323 h.connect(host)
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
1324 h.putrequest('GET', selector)
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
1325 h.endheaders()
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1326 status, reason, headers = h.getreply()
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1327 print 'status =', status
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1328 print 'reason =', reason
24168
fdd836575248 Fix for SF bug 579107.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24123
diff changeset
1329 print "read", len(h.getfile().read())
6787
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
1330 print
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
1331 if headers:
15582
6179d4afa4e4 Get rid of string functions.
Guido van Rossum <guido@python.org>
parents: 15503
diff changeset
1332 for header in headers.headers: print header.strip()
6787
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
1333 print
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1334
22192
044f9e71320f SF bug report #405939: wrong Host header with proxy
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 21962
diff changeset
1335 # minimal test that code to extract host from url works
044f9e71320f SF bug report #405939: wrong Host header with proxy
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 21962
diff changeset
1336 class HTTP11(HTTP):
044f9e71320f SF bug report #405939: wrong Host header with proxy
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 21962
diff changeset
1337 _http_vsn = 11
044f9e71320f SF bug report #405939: wrong Host header with proxy
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 21962
diff changeset
1338 _http_vsn_str = 'HTTP/1.1'
044f9e71320f SF bug report #405939: wrong Host header with proxy
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 21962
diff changeset
1339
044f9e71320f SF bug report #405939: wrong Host header with proxy
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 21962
diff changeset
1340 h = HTTP11('www.python.org')
044f9e71320f SF bug report #405939: wrong Host header with proxy
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 21962
diff changeset
1341 h.putrequest('GET', 'http://www.python.org/~jeremy/')
044f9e71320f SF bug report #405939: wrong Host header with proxy
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 21962
diff changeset
1342 h.endheaders()
044f9e71320f SF bug report #405939: wrong Host header with proxy
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 21962
diff changeset
1343 h.getreply()
044f9e71320f SF bug report #405939: wrong Host header with proxy
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 21962
diff changeset
1344 h.close()
044f9e71320f SF bug report #405939: wrong Host header with proxy
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 21962
diff changeset
1345
43008
4771639fc049 This contains a number of things:
Bill Janssen <janssen@parc.com>
parents: 42385
diff changeset
1346 try:
4771639fc049 This contains a number of things:
Bill Janssen <janssen@parc.com>
parents: 42385
diff changeset
1347 import ssl
4771639fc049 This contains a number of things:
Bill Janssen <janssen@parc.com>
parents: 42385
diff changeset
1348 except ImportError:
4771639fc049 This contains a number of things:
Bill Janssen <janssen@parc.com>
parents: 42385
diff changeset
1349 pass
4771639fc049 This contains a number of things:
Bill Janssen <janssen@parc.com>
parents: 42385
diff changeset
1350 else:
24263
e6033254df21 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 24262
diff changeset
1351
24168
fdd836575248 Fix for SF bug 579107.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24123
diff changeset
1352 for host, selector in (('sourceforge.net', '/projects/python'),
fdd836575248 Fix for SF bug 579107.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24123
diff changeset
1353 ):
fdd836575248 Fix for SF bug 579107.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24123
diff changeset
1354 print "https://%s%s" % (host, selector)
fdd836575248 Fix for SF bug 579107.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24123
diff changeset
1355 hs = HTTPS()
24262
3b571e32a6e4 Send HTTP requests with a single send() call instead of many.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24216
diff changeset
1356 hs.set_debuglevel(dl)
24168
fdd836575248 Fix for SF bug 579107.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24123
diff changeset
1357 hs.connect(host)
fdd836575248 Fix for SF bug 579107.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24123
diff changeset
1358 hs.putrequest('GET', selector)
fdd836575248 Fix for SF bug 579107.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24123
diff changeset
1359 hs.endheaders()
fdd836575248 Fix for SF bug 579107.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24123
diff changeset
1360 status, reason, headers = hs.getreply()
fdd836575248 Fix for SF bug 579107.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24123
diff changeset
1361 print 'status =', status
fdd836575248 Fix for SF bug 579107.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24123
diff changeset
1362 print 'reason =', reason
fdd836575248 Fix for SF bug 579107.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24123
diff changeset
1363 print "read", len(hs.getfile().read())
fdd836575248 Fix for SF bug 579107.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24123
diff changeset
1364 print
fdd836575248 Fix for SF bug 579107.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24123
diff changeset
1365 if headers:
fdd836575248 Fix for SF bug 579107.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24123
diff changeset
1366 for header in headers.headers: print header.strip()
fdd836575248 Fix for SF bug 579107.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24123
diff changeset
1367 print
1555
6d35308261f4 Towards a standard access mechanism for URLs.
Guido van Rossum <guido@python.org>
parents:
diff changeset
1368
6d35308261f4 Towards a standard access mechanism for URLs.
Guido van Rossum <guido@python.org>
parents:
diff changeset
1369 if __name__ == '__main__':
6787
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
1370 test()