Mercurial > cpython-lru_cache-weakref

annotate Lib/http/client.py @ 66807:d28126ebd60d

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Entries for datetime, callable, and collections.Counter.
author Raymond Hettinger <python@rcn.com>
date Thu, 09 Dec 2010 16:41:54 +0000
parents c6b2f417830e
children b6d3fe70802f
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1 """HTTP/1.1 client library
6787
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
2
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
3 <intro stuff goes here>
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
4 <other stuff, too>
6787
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
5
39442
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
6 HTTPConnection goes through a number of "states", which define when a client
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
7 may legally make another request or fetch the response for a particular
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
8 request. This diagram details these state transitions:
1555
6d35308261f4 Towards a standard access mechanism for URLs.
Guido van Rossum <guido@python.org>
parents:
diff changeset
9
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
10 (null)
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
11 |
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
12 | HTTPConnection()
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
13 v
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
14 Idle
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
15 |
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
16 | putrequest()
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
17 v
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
18 Request-started
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
19 |
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
20 | ( putheader() )* endheaders()
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
21 v
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
22 Request-sent
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
23 |
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
24 | response = getresponse()
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
25 v
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
26 Unread-response [Response-headers-read]
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
27 |\____________________
15845
210bbe53b6d4 No text file relying on significant trailing whitespace is robust under
Tim Peters <tim.peters@gmail.com>
parents: 15844
diff changeset
28 | |
210bbe53b6d4 No text file relying on significant trailing whitespace is robust under
Tim Peters <tim.peters@gmail.com>
parents: 15844
diff changeset
29 | response.read() | putrequest()
210bbe53b6d4 No text file relying on significant trailing whitespace is robust under
Tim Peters <tim.peters@gmail.com>
parents: 15844
diff changeset
30 v v
210bbe53b6d4 No text file relying on significant trailing whitespace is robust under
Tim Peters <tim.peters@gmail.com>
parents: 15844
diff changeset
31 Idle Req-started-unread-response
210bbe53b6d4 No text file relying on significant trailing whitespace is robust under
Tim Peters <tim.peters@gmail.com>
parents: 15844
diff changeset
32 ______/|
210bbe53b6d4 No text file relying on significant trailing whitespace is robust under
Tim Peters <tim.peters@gmail.com>
parents: 15844
diff changeset
33 / |
210bbe53b6d4 No text file relying on significant trailing whitespace is robust under
Tim Peters <tim.peters@gmail.com>
parents: 15844
diff changeset
34 response.read() | | ( putheader() )* endheaders()
210bbe53b6d4 No text file relying on significant trailing whitespace is robust under
Tim Peters <tim.peters@gmail.com>
parents: 15844
diff changeset
35 v v
210bbe53b6d4 No text file relying on significant trailing whitespace is robust under
Tim Peters <tim.peters@gmail.com>
parents: 15844
diff changeset
36 Request-started Req-sent-unread-response
210bbe53b6d4 No text file relying on significant trailing whitespace is robust under
Tim Peters <tim.peters@gmail.com>
parents: 15844
diff changeset
37 |
210bbe53b6d4 No text file relying on significant trailing whitespace is robust under
Tim Peters <tim.peters@gmail.com>
parents: 15844
diff changeset
38 | response.read()
210bbe53b6d4 No text file relying on significant trailing whitespace is robust under
Tim Peters <tim.peters@gmail.com>
parents: 15844
diff changeset
39 v
210bbe53b6d4 No text file relying on significant trailing whitespace is robust under
Tim Peters <tim.peters@gmail.com>
parents: 15844
diff changeset
40 Request-sent
6787
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
41
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
42 This diagram presents the following rules:
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
43 -- a second request may not be started until {response-headers-read}
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
44 -- a response [object] cannot be retrieved until {request-sent}
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
45 -- there is no differentiation between an unread response body and a
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
46 partially read response body
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
47
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
48 Note: this enforcement is applied by the HTTPConnection class. The
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
49 HTTPResponse class does not enforce this state machine, which
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
50 implies sophisticated clients may accelerate the request/response
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
51 pipeline. Caution should be taken, though: accelerating the states
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
52 beyond the above pattern may imply knowledge of the server's
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
53 connection-close behavior for certain requests. For example, it
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
54 is impossible to tell whether the server will close the connection
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
55 UNTIL the response headers have been read; this means that further
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
56 requests cannot be placed into the pipeline until it is known that
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
57 the server will NOT be closing the connection.
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
58
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
59 Logical State __state __response
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
60 ------------- ------- ----------
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
61 Idle _CS_IDLE None
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
62 Request-started _CS_REQ_STARTED None
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
63 Request-sent _CS_REQ_SENT None
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
64 Unread-response _CS_IDLE <response_class>
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
65 Req-started-unread-response _CS_REQ_STARTED <response_class>
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
66 Req-sent-unread-response _CS_REQ_SENT <response_class>
6787
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
67 """
1555
6d35308261f4 Towards a standard access mechanism for URLs.
Guido van Rossum <guido@python.org>
parents:
diff changeset
68
47635
f31e32a2013c Patch for issue 2848, mostly by Humberto Diogenes, with a couple of
Barry Warsaw <barry@python.org>
parents: 47559
diff changeset
69 import email.parser
f31e32a2013c Patch for issue 2848, mostly by Humberto Diogenes, with a couple of
Barry Warsaw <barry@python.org>
parents: 47559
diff changeset
70 import email.message
52375
a726f7928561 Remove special logic that closes HTTPConnection socket on EPIPE.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52370
diff changeset
71 import io
a726f7928561 Remove special logic that closes HTTPConnection socket on EPIPE.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52370
diff changeset
72 import os
a726f7928561 Remove special logic that closes HTTPConnection socket on EPIPE.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52370
diff changeset
73 import socket
48334
5a416a6417d3 Make a new urllib package .
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 47635
diff changeset
74 from urllib.parse import urlsplit
43292
876b7ef4775d Merged revisions 57778-58052 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 43095
diff changeset
75 import warnings
1555
6d35308261f4 Towards a standard access mechanism for URLs.
Guido van Rossum <guido@python.org>
parents:
diff changeset
76
43095
4a07a288f5c4 Merged revisions 57620-57771 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 42964
diff changeset
77 __all__ = ["HTTPResponse", "HTTPConnection",
18013
99aea6f5cebd more public symbols for __all__
Skip Montanaro <skip@pobox.com>
parents: 17482
diff changeset
78 "HTTPException", "NotConnected", "UnknownProtocol",
24042
63fbbed10714 Simplify HTTPSConnection constructor.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24038
diff changeset
79 "UnknownTransferEncoding", "UnimplementedFileMode",
63fbbed10714 Simplify HTTPSConnection constructor.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24038
diff changeset
80 "IncompleteRead", "InvalidURL", "ImproperConnectionState",
63fbbed10714 Simplify HTTPSConnection constructor.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24038
diff changeset
81 "CannotSendRequest", "CannotSendHeader", "ResponseNotReady",
36559
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
82 "BadStatusLine", "error", "responses"]
16160
46920b60cd33 added a few more __all__ lists
Skip Montanaro <skip@pobox.com>
parents: 15878
diff changeset
83
1555
6d35308261f4 Towards a standard access mechanism for URLs.
Guido van Rossum <guido@python.org>
parents:
diff changeset
84 HTTP_PORT = 80
11527
28b8585354de OpenSSL support. This is based on patches for a version of SSLeay by
Guido van Rossum <guido@python.org>
parents: 7122
diff changeset
85 HTTPS_PORT = 443
28b8585354de OpenSSL support. This is based on patches for a version of SSLeay by
Guido van Rossum <guido@python.org>
parents: 7122
diff changeset
86
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
87 _UNKNOWN = 'UNKNOWN'
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
88
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
89 # connection states
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
90 _CS_IDLE = 'Idle'
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
91 _CS_REQ_STARTED = 'Request-started'
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
92 _CS_REQ_SENT = 'Request-sent'
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
93
33506
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
94 # status codes
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
95 # informational
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
96 CONTINUE = 100
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
97 SWITCHING_PROTOCOLS = 101
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
98 PROCESSING = 102
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
99
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
100 # successful
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
101 OK = 200
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
102 CREATED = 201
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
103 ACCEPTED = 202
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
104 NON_AUTHORITATIVE_INFORMATION = 203
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
105 NO_CONTENT = 204
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
106 RESET_CONTENT = 205
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
107 PARTIAL_CONTENT = 206
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
108 MULTI_STATUS = 207
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
109 IM_USED = 226
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
110
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
111 # redirection
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
112 MULTIPLE_CHOICES = 300
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
113 MOVED_PERMANENTLY = 301
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
114 FOUND = 302
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
115 SEE_OTHER = 303
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
116 NOT_MODIFIED = 304
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
117 USE_PROXY = 305
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
118 TEMPORARY_REDIRECT = 307
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
119
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
120 # client error
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
121 BAD_REQUEST = 400
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
122 UNAUTHORIZED = 401
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
123 PAYMENT_REQUIRED = 402
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
124 FORBIDDEN = 403
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
125 NOT_FOUND = 404
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
126 METHOD_NOT_ALLOWED = 405
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
127 NOT_ACCEPTABLE = 406
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
128 PROXY_AUTHENTICATION_REQUIRED = 407
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
129 REQUEST_TIMEOUT = 408
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
130 CONFLICT = 409
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
131 GONE = 410
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
132 LENGTH_REQUIRED = 411
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
133 PRECONDITION_FAILED = 412
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
134 REQUEST_ENTITY_TOO_LARGE = 413
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
135 REQUEST_URI_TOO_LONG = 414
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
136 UNSUPPORTED_MEDIA_TYPE = 415
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
137 REQUESTED_RANGE_NOT_SATISFIABLE = 416
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
138 EXPECTATION_FAILED = 417
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
139 UNPROCESSABLE_ENTITY = 422
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
140 LOCKED = 423
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
141 FAILED_DEPENDENCY = 424
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
142 UPGRADE_REQUIRED = 426
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
143
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
144 # server error
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
145 INTERNAL_SERVER_ERROR = 500
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
146 NOT_IMPLEMENTED = 501
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
147 BAD_GATEWAY = 502
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
148 SERVICE_UNAVAILABLE = 503
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
149 GATEWAY_TIMEOUT = 504
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
150 HTTP_VERSION_NOT_SUPPORTED = 505
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
151 INSUFFICIENT_STORAGE = 507
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
152 NOT_EXTENDED = 510
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
153
36559
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
154 # Mapping status codes to official W3C names
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
155 responses = {
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
156 100: 'Continue',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
157 101: 'Switching Protocols',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
158
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
159 200: 'OK',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
160 201: 'Created',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
161 202: 'Accepted',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
162 203: 'Non-Authoritative Information',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
163 204: 'No Content',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
164 205: 'Reset Content',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
165 206: 'Partial Content',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
166
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
167 300: 'Multiple Choices',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
168 301: 'Moved Permanently',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
169 302: 'Found',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
170 303: 'See Other',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
171 304: 'Not Modified',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
172 305: 'Use Proxy',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
173 306: '(Unused)',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
174 307: 'Temporary Redirect',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
175
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
176 400: 'Bad Request',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
177 401: 'Unauthorized',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
178 402: 'Payment Required',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
179 403: 'Forbidden',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
180 404: 'Not Found',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
181 405: 'Method Not Allowed',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
182 406: 'Not Acceptable',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
183 407: 'Proxy Authentication Required',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
184 408: 'Request Timeout',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
185 409: 'Conflict',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
186 410: 'Gone',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
187 411: 'Length Required',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
188 412: 'Precondition Failed',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
189 413: 'Request Entity Too Large',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
190 414: 'Request-URI Too Long',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
191 415: 'Unsupported Media Type',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
192 416: 'Requested Range Not Satisfiable',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
193 417: 'Expectation Failed',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
194
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
195 500: 'Internal Server Error',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
196 501: 'Not Implemented',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
197 502: 'Bad Gateway',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
198 503: 'Service Unavailable',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
199 504: 'Gateway Timeout',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
200 505: 'HTTP Version Not Supported',
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
201 }
09230357f927 Move "httpresponses" dict from urllib2 to httplib where it belongs.
Georg Brandl <georg@python.org>
parents: 35652
diff changeset
202
35652
6a20831d357d bug [ 1296004 ] MemoryError in httplib
Georg Brandl <georg@python.org>
parents: 35158
diff changeset
203 # maximal amount of data to read at one time in _safe_read
6a20831d357d bug [ 1296004 ] MemoryError in httplib
Georg Brandl <georg@python.org>
parents: 35158
diff changeset
204 MAXAMOUNT = 1048576
6a20831d357d bug [ 1296004 ] MemoryError in httplib
Georg Brandl <georg@python.org>
parents: 35158
diff changeset
205
47635
f31e32a2013c Patch for issue 2848, mostly by Humberto Diogenes, with a couple of
Barry Warsaw <barry@python.org>
parents: 47559
diff changeset
206 class HTTPMessage(email.message.Message):
52522
9eceb618274a Simplify the Request class. The basic components of the parsed
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52375
diff changeset
207 # XXX The only usage of this method is in
9eceb618274a Simplify the Request class. The basic components of the parsed
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52375
diff changeset
208 # http.server.CGIHTTPRequestHandler. Maybe move the code there so
9eceb618274a Simplify the Request class. The basic components of the parsed
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52375
diff changeset
209 # that it doesn't need to be part of the public API. The API has
9eceb618274a Simplify the Request class. The basic components of the parsed
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52375
diff changeset
210 # never been defined so this could cause backwards compatibility
9eceb618274a Simplify the Request class. The basic components of the parsed
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52375
diff changeset
211 # issues.
9eceb618274a Simplify the Request class. The basic components of the parsed
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52375
diff changeset
212
47635
f31e32a2013c Patch for issue 2848, mostly by Humberto Diogenes, with a couple of
Barry Warsaw <barry@python.org>
parents: 47559
diff changeset
213 def getallmatchingheaders(self, name):
f31e32a2013c Patch for issue 2848, mostly by Humberto Diogenes, with a couple of
Barry Warsaw <barry@python.org>
parents: 47559
diff changeset
214 """Find all header lines matching a given header name.
24123
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
215
47635
f31e32a2013c Patch for issue 2848, mostly by Humberto Diogenes, with a couple of
Barry Warsaw <barry@python.org>
parents: 47559
diff changeset
216 Look through the list of headers and find all lines matching a given
f31e32a2013c Patch for issue 2848, mostly by Humberto Diogenes, with a couple of
Barry Warsaw <barry@python.org>
parents: 47559
diff changeset
217 header name (and their continuation lines). A list of the lines is
f31e32a2013c Patch for issue 2848, mostly by Humberto Diogenes, with a couple of
Barry Warsaw <barry@python.org>
parents: 47559
diff changeset
218 returned, without interpretation. If the header does not occur, an
f31e32a2013c Patch for issue 2848, mostly by Humberto Diogenes, with a couple of
Barry Warsaw <barry@python.org>
parents: 47559
diff changeset
219 empty list is returned. If the header occurs multiple times, all
f31e32a2013c Patch for issue 2848, mostly by Humberto Diogenes, with a couple of
Barry Warsaw <barry@python.org>
parents: 47559
diff changeset
220 occurrences are returned. Case is not important in the header name.
24123
aa6217bf9776 Fix for SF bug #432621: httplib: multiple Set-Cookie headers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24120
diff changeset
221
47635
f31e32a2013c Patch for issue 2848, mostly by Humberto Diogenes, with a couple of
Barry Warsaw <barry@python.org>
parents: 47559
diff changeset
222 """
f31e32a2013c Patch for issue 2848, mostly by Humberto Diogenes, with a couple of
Barry Warsaw <barry@python.org>
parents: 47559
diff changeset
223 name = name.lower() + ':'
f31e32a2013c Patch for issue 2848, mostly by Humberto Diogenes, with a couple of
Barry Warsaw <barry@python.org>
parents: 47559
diff changeset
224 n = len(name)
f31e32a2013c Patch for issue 2848, mostly by Humberto Diogenes, with a couple of
Barry Warsaw <barry@python.org>
parents: 47559
diff changeset
225 lst = []
f31e32a2013c Patch for issue 2848, mostly by Humberto Diogenes, with a couple of
Barry Warsaw <barry@python.org>
parents: 47559
diff changeset
226 hit = 0
f31e32a2013c Patch for issue 2848, mostly by Humberto Diogenes, with a couple of
Barry Warsaw <barry@python.org>
parents: 47559
diff changeset
227 for line in self.keys():
f31e32a2013c Patch for issue 2848, mostly by Humberto Diogenes, with a couple of
Barry Warsaw <barry@python.org>
parents: 47559
diff changeset
228 if line[:n].lower() == name:
f31e32a2013c Patch for issue 2848, mostly by Humberto Diogenes, with a couple of
Barry Warsaw <barry@python.org>
parents: 47559
diff changeset
229 hit = 1
f31e32a2013c Patch for issue 2848, mostly by Humberto Diogenes, with a couple of
Barry Warsaw <barry@python.org>
parents: 47559
diff changeset
230 elif not line[:1].isspace():
f31e32a2013c Patch for issue 2848, mostly by Humberto Diogenes, with a couple of
Barry Warsaw <barry@python.org>
parents: 47559
diff changeset
231 hit = 0
f31e32a2013c Patch for issue 2848, mostly by Humberto Diogenes, with a couple of
Barry Warsaw <barry@python.org>
parents: 47559
diff changeset
232 if hit:
f31e32a2013c Patch for issue 2848, mostly by Humberto Diogenes, with a couple of
Barry Warsaw <barry@python.org>
parents: 47559
diff changeset
233 lst.append(line)
f31e32a2013c Patch for issue 2848, mostly by Humberto Diogenes, with a couple of
Barry Warsaw <barry@python.org>
parents: 47559
diff changeset
234 return lst
f31e32a2013c Patch for issue 2848, mostly by Humberto Diogenes, with a couple of
Barry Warsaw <barry@python.org>
parents: 47559
diff changeset
235
52369
8d6918af232b Fix compatibility issue with HTTPMessage class.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52367
diff changeset
236 def parse_headers(fp, _class=HTTPMessage):
47635
f31e32a2013c Patch for issue 2848, mostly by Humberto Diogenes, with a couple of
Barry Warsaw <barry@python.org>
parents: 47559
diff changeset
237 """Parses only RFC2822 headers from a file pointer.
f31e32a2013c Patch for issue 2848, mostly by Humberto Diogenes, with a couple of
Barry Warsaw <barry@python.org>
parents: 47559
diff changeset
238
f31e32a2013c Patch for issue 2848, mostly by Humberto Diogenes, with a couple of
Barry Warsaw <barry@python.org>
parents: 47559
diff changeset
239 email Parser wants to see strings rather than bytes.
f31e32a2013c Patch for issue 2848, mostly by Humberto Diogenes, with a couple of
Barry Warsaw <barry@python.org>
parents: 47559
diff changeset
240 But a TextIOWrapper around self.rfile would buffer too many bytes
f31e32a2013c Patch for issue 2848, mostly by Humberto Diogenes, with a couple of
Barry Warsaw <barry@python.org>
parents: 47559
diff changeset
241 from the stream, bytes which we later need to read as bytes.
f31e32a2013c Patch for issue 2848, mostly by Humberto Diogenes, with a couple of
Barry Warsaw <barry@python.org>
parents: 47559
diff changeset
242 So we read the correct bytes here, as bytes, for email Parser
f31e32a2013c Patch for issue 2848, mostly by Humberto Diogenes, with a couple of
Barry Warsaw <barry@python.org>
parents: 47559
diff changeset
243 to parse.
f31e32a2013c Patch for issue 2848, mostly by Humberto Diogenes, with a couple of
Barry Warsaw <barry@python.org>
parents: 47559
diff changeset
244
f31e32a2013c Patch for issue 2848, mostly by Humberto Diogenes, with a couple of
Barry Warsaw <barry@python.org>
parents: 47559
diff changeset
245 """
f31e32a2013c Patch for issue 2848, mostly by Humberto Diogenes, with a couple of
Barry Warsaw <barry@python.org>
parents: 47559
diff changeset
246 headers = []
f31e32a2013c Patch for issue 2848, mostly by Humberto Diogenes, with a couple of
Barry Warsaw <barry@python.org>
parents: 47559
diff changeset
247 while True:
f31e32a2013c Patch for issue 2848, mostly by Humberto Diogenes, with a couple of
Barry Warsaw <barry@python.org>
parents: 47559
diff changeset
248 line = fp.readline()
f31e32a2013c Patch for issue 2848, mostly by Humberto Diogenes, with a couple of
Barry Warsaw <barry@python.org>
parents: 47559
diff changeset
249 headers.append(line)
f31e32a2013c Patch for issue 2848, mostly by Humberto Diogenes, with a couple of
Barry Warsaw <barry@python.org>
parents: 47559
diff changeset
250 if line in (b'\r\n', b'\n', b''):
f31e32a2013c Patch for issue 2848, mostly by Humberto Diogenes, with a couple of
Barry Warsaw <barry@python.org>
parents: 47559
diff changeset
251 break
f31e32a2013c Patch for issue 2848, mostly by Humberto Diogenes, with a couple of
Barry Warsaw <barry@python.org>
parents: 47559
diff changeset
252 hstring = b''.join(headers).decode('iso-8859-1')
52369
8d6918af232b Fix compatibility issue with HTTPMessage class.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52367
diff changeset
253 return email.parser.Parser(_class=_class).parsestr(hstring)
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
254
51532
af469dc15fa7 Issue #4631: Fix urlopen() result when an HTTP response uses chunked encoding.
Antoine Pitrou <solipsis@pitrou.net>
parents: 51449
diff changeset
255 class HTTPResponse(io.RawIOBase):
24119
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
256
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
257 # strict: If true, raise BadStatusLine if the status line can't be
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
258 # parsed as a valid HTTP/1.0 or 1.1 status line. By default it is
24378
Skip Montanaro <skip@pobox.com>
parents: 24264
diff changeset
259 # false because it prevents clients from talking to HTTP/0.9
24119
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
260 # servers. Note that a response with a sufficiently corrupted
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
261 # status line will look like an HTTP/0.9 response.
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
262
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
263 # See RFC 2616 sec 19.6 and RFC 1945 sec 6 for details.
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
264
42425
7794da1334a0 Fix status line parsing for http response.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 42133
diff changeset
265 # The bytes from the socket object are iso-8859-1 strings.
7794da1334a0 Fix status line parsing for http response.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 42133
diff changeset
266 # See RFC 2616 sec 2.2 which notes an exception for MIME-encoded
7794da1334a0 Fix status line parsing for http response.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 42133
diff changeset
267 # text following RFC 2047. The basic status line parsing only
7794da1334a0 Fix status line parsing for http response.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 42133
diff changeset
268 # accepts iso-8859-1.
7794da1334a0 Fix status line parsing for http response.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 42133
diff changeset
269
52522
9eceb618274a Simplify the Request class. The basic components of the parsed
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52375
diff changeset
270 def __init__(self, sock, debuglevel=0, strict=0, method=None, url=None):
9eceb618274a Simplify the Request class. The basic components of the parsed
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52375
diff changeset
271 # If the response includes a content-length header, we need to
9eceb618274a Simplify the Request class. The basic components of the parsed
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52375
diff changeset
272 # make sure that the client doesn't read more than the
42452
f4409c5d7994 Change urllib to use HTTPConnection rather than old HTTP class.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 42444
diff changeset
273 # specified number of bytes. If it does, it will block until
52522
9eceb618274a Simplify the Request class. The basic components of the parsed
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52375
diff changeset
274 # the server times out and closes the connection. This will
9eceb618274a Simplify the Request class. The basic components of the parsed
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52375
diff changeset
275 # happen if a self.fp.read() is done (without a size) whether
9eceb618274a Simplify the Request class. The basic components of the parsed
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52375
diff changeset
276 # self.fp is buffered or not. So, no self.fp.read() by
9eceb618274a Simplify the Request class. The basic components of the parsed
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52375
diff changeset
277 # clients unless they know what they are doing.
51449
ccc7188fdd28 revert 69416; it didn't help much
Benjamin Peterson <benjamin@python.org>
parents: 51447
diff changeset
278 self.fp = sock.makefile("rb")
14672
1b20ad6eb501 Do not close socket when a Content-Length is 0. This make the
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14560
diff changeset
279 self.debuglevel = debuglevel
24119
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
280 self.strict = strict
28978
83fe0f5c9359 SF bug 622042: Don't expect response body from HEAD request.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 28671
diff changeset
281 self._method = method
11527
28b8585354de OpenSSL support. This is based on patches for a version of SSLeay by
Guido van Rossum <guido@python.org>
parents: 7122
diff changeset
282
52522
9eceb618274a Simplify the Request class. The basic components of the parsed
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52375
diff changeset
283 # The HTTPResponse object is returned via urllib. The clients
9eceb618274a Simplify the Request class. The basic components of the parsed
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52375
diff changeset
284 # of http and urllib expect different attributes for the
9eceb618274a Simplify the Request class. The basic components of the parsed
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52375
diff changeset
285 # headers. headers is used here and supports urllib. msg is
9eceb618274a Simplify the Request class. The basic components of the parsed
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52375
diff changeset
286 # provided as a backwards compatibility layer for http
9eceb618274a Simplify the Request class. The basic components of the parsed
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52375
diff changeset
287 # clients.
9eceb618274a Simplify the Request class. The basic components of the parsed
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52375
diff changeset
288
9eceb618274a Simplify the Request class. The basic components of the parsed
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52375
diff changeset
289 self.headers = self.msg = None
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
290
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
291 # from the Status-Line of the response
15868
3060cbfa01be Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15859
diff changeset
292 self.version = _UNKNOWN # HTTP-Version
3060cbfa01be Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15859
diff changeset
293 self.status = _UNKNOWN # Status-Code
3060cbfa01be Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15859
diff changeset
294 self.reason = _UNKNOWN # Reason-Phrase
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
295
15868
3060cbfa01be Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15859
diff changeset
296 self.chunked = _UNKNOWN # is "chunked" being used?
3060cbfa01be Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15859
diff changeset
297 self.chunk_left = _UNKNOWN # bytes left to read in current chunk
3060cbfa01be Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15859
diff changeset
298 self.length = _UNKNOWN # number of bytes left in response
3060cbfa01be Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15859
diff changeset
299 self.will_close = _UNKNOWN # conn will close at end of response
11527
28b8585354de OpenSSL support. This is based on patches for a version of SSLeay by
Guido van Rossum <guido@python.org>
parents: 7122
diff changeset
300
24038
9377158652dd Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23620
diff changeset
301 def _read_status(self):
42444
12dbaecfb44b Make sure LineAndFileWrapper gets bytes() as its first argument.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 42443
diff changeset
302 # Initialize with Simple-Response defaults.
42425
7794da1334a0 Fix status line parsing for http response.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 42133
diff changeset
303 line = str(self.fp.readline(), "iso-8859-1")
14672
1b20ad6eb501 Do not close socket when a Content-Length is 0. This make the
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14560
diff changeset
304 if self.debuglevel > 0:
40775
4cdf21a0dbbc Fix most trivially-findable print statements.
Guido van Rossum <guido@python.org>
parents: 40742
diff changeset
305 print("reply:", repr(line))
29688
9fe67f7f3459 Fix sf bug 666219: assertion error in httplib.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 29584
diff changeset
306 if not line:
9fe67f7f3459 Fix sf bug 666219: assertion error in httplib.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 29584
diff changeset
307 # Presumably, the server closed the connection before
9fe67f7f3459 Fix sf bug 666219: assertion error in httplib.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 29584
diff changeset
308 # sending a valid response.
9fe67f7f3459 Fix sf bug 666219: assertion error in httplib.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 29584
diff changeset
309 raise BadStatusLine(line)
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
310 try:
15582
6179d4afa4e4 Get rid of string functions.
Guido van Rossum <guido@python.org>
parents: 15503
diff changeset
311 [version, status, reason] = line.split(None, 2)
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
312 except ValueError:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
313 try:
15582
6179d4afa4e4 Get rid of string functions.
Guido van Rossum <guido@python.org>
parents: 15503
diff changeset
314 [version, status] = line.split(None, 1)
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
315 reason = ""
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
316 except ValueError:
24119
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
317 # empty version will cause next test to fail and status
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
318 # will be treated as 0.9 response.
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
319 version = ""
42426
ae72bf5ae465 Make consistent use of "" for string literals in new classes.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 42425
diff changeset
320 if not version.startswith("HTTP/"):
24119
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
321 if self.strict:
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
322 self.close()
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
323 raise BadStatusLine(line)
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
324 else:
42444
12dbaecfb44b Make sure LineAndFileWrapper gets bytes() as its first argument.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 42443
diff changeset
325 # Assume it's a Simple-Response from an 0.9 server.
12dbaecfb44b Make sure LineAndFileWrapper gets bytes() as its first argument.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 42443
diff changeset
326 # We have to convert the first line back to raw bytes
12dbaecfb44b Make sure LineAndFileWrapper gets bytes() as its first argument.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 42443
diff changeset
327 # because self.fp.readline() needs to return bytes.
42964
a7951abb8534 Make test_urllib be strict about str/bytes.
Guido van Rossum <guido@python.org>
parents: 42590
diff changeset
328 self.fp = LineAndFileWrapper(bytes(line, "ascii"), self.fp)
24119
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
329 return "HTTP/0.9", 200, ""
11527
28b8585354de OpenSSL support. This is based on patches for a version of SSLeay by
Guido van Rossum <guido@python.org>
parents: 7122
diff changeset
330
17482
2265032efc9f SF patch #405845 by Martin von Löwis
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 16400
diff changeset
331 # The status code is a three-digit number
2265032efc9f SF patch #405845 by Martin von Löwis
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 16400
diff changeset
332 try:
24038
9377158652dd Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23620
diff changeset
333 status = int(status)
17482
2265032efc9f SF patch #405845 by Martin von Löwis
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 16400
diff changeset
334 if status < 100 or status > 999:
2265032efc9f SF patch #405845 by Martin von Löwis
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 16400
diff changeset
335 raise BadStatusLine(line)
2265032efc9f SF patch #405845 by Martin von Löwis
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 16400
diff changeset
336 except ValueError:
2265032efc9f SF patch #405845 by Martin von Löwis
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 16400
diff changeset
337 raise BadStatusLine(line)
24038
9377158652dd Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23620
diff changeset
338 return version, status, reason
9377158652dd Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23620
diff changeset
339
24216
c8a19a428358 Change _begin() back to begin().
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24168
diff changeset
340 def begin(self):
52522
9eceb618274a Simplify the Request class. The basic components of the parsed
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52375
diff changeset
341 if self.headers is not None:
24038
9377158652dd Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23620
diff changeset
342 # we've already started reading the response
9377158652dd Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23620
diff changeset
343 return
9377158652dd Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23620
diff changeset
344
9377158652dd Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23620
diff changeset
345 # read until we get a non-100 response
28083
29965d0d4650 Module review:
Raymond Hettinger <python@rcn.com>
parents: 28063
diff changeset
346 while True:
24038
9377158652dd Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23620
diff changeset
347 version, status, reason = self._read_status()
33506
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
348 if status != CONTINUE:
24038
9377158652dd Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23620
diff changeset
349 break
9377158652dd Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23620
diff changeset
350 # skip the header from the 100 response
28083
29965d0d4650 Module review:
Raymond Hettinger <python@rcn.com>
parents: 28063
diff changeset
351 while True:
24038
9377158652dd Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23620
diff changeset
352 skip = self.fp.readline().strip()
9377158652dd Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23620
diff changeset
353 if not skip:
9377158652dd Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23620
diff changeset
354 break
9377158652dd Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23620
diff changeset
355 if self.debuglevel > 0:
40775
4cdf21a0dbbc Fix most trivially-findable print statements.
Guido van Rossum <guido@python.org>
parents: 40742
diff changeset
356 print("header:", skip)
24263
e6033254df21 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 24262
diff changeset
357
52522
9eceb618274a Simplify the Request class. The basic components of the parsed
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52375
diff changeset
358 self.code = self.status = status
15582
6179d4afa4e4 Get rid of string functions.
Guido van Rossum <guido@python.org>
parents: 15503
diff changeset
359 self.reason = reason.strip()
42426
ae72bf5ae465 Make consistent use of "" for string literals in new classes.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 42425
diff changeset
360 if version == "HTTP/1.0":
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
361 self.version = 10
42426
ae72bf5ae465 Make consistent use of "" for string literals in new classes.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 42425
diff changeset
362 elif version.startswith("HTTP/1."):
15868
3060cbfa01be Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15859
diff changeset
363 self.version = 11 # use HTTP/1.1 code for HTTP/1.x where x>=1
42426
ae72bf5ae465 Make consistent use of "" for string literals in new classes.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 42425
diff changeset
364 elif version == "HTTP/0.9":
15182
b3387eeba68d If the status line is invalid, assume it is a pre-1.0 response. The
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14740
diff changeset
365 self.version = 9
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
366 else:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
367 raise UnknownProtocol(version)
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
368
15182
b3387eeba68d If the status line is invalid, assume it is a pre-1.0 response. The
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14740
diff changeset
369 if self.version == 9:
35158
daebfde0e1cf backport bug [ 1155638 ] self.length shield exception in httplib
Georg Brandl <georg@python.org>
parents: 33933
diff changeset
370 self.length = None
50540
4229a4b14f1d Merged revisions 67653,67655,67682-67683,67724,67755,67780-67783,67786,67789,67841,67843,67865,67881,67884,67925,67929,67931 via svnmerge from
Georg Brandl <georg@python.org>
parents: 49907
diff changeset
371 self.chunked = False
4229a4b14f1d Merged revisions 67653,67655,67682-67683,67724,67755,67780-67783,67786,67789,67841,67843,67865,67881,67884,67925,67929,67931 via svnmerge from
Georg Brandl <georg@python.org>
parents: 49907
diff changeset
372 self.will_close = True
52522
9eceb618274a Simplify the Request class. The basic components of the parsed
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52375
diff changeset
373 self.headers = self.msg = email.message_from_string('')
15182
b3387eeba68d If the status line is invalid, assume it is a pre-1.0 response. The
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14740
diff changeset
374 return
b3387eeba68d If the status line is invalid, assume it is a pre-1.0 response. The
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14740
diff changeset
375
52522
9eceb618274a Simplify the Request class. The basic components of the parsed
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52375
diff changeset
376 self.headers = self.msg = parse_headers(self.fp)
47635
f31e32a2013c Patch for issue 2848, mostly by Humberto Diogenes, with a couple of
Barry Warsaw <barry@python.org>
parents: 47559
diff changeset
377
14672
1b20ad6eb501 Do not close socket when a Content-Length is 0. This make the
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14560
diff changeset
378 if self.debuglevel > 0:
52522
9eceb618274a Simplify the Request class. The basic components of the parsed
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52375
diff changeset
379 for hdr in self.headers:
42426
ae72bf5ae465 Make consistent use of "" for string literals in new classes.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 42425
diff changeset
380 print("header:", hdr, end=" ")
1555
6d35308261f4 Towards a standard access mechanism for URLs.
Guido van Rossum <guido@python.org>
parents:
diff changeset
381
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
382 # are we using the chunked-style of transfer encoding?
52522
9eceb618274a Simplify the Request class. The basic components of the parsed
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52375
diff changeset
383 tr_enc = self.headers.get("transfer-encoding")
25107
0ecb3cabf40b SF bug [ 600488 ] Robustness tweak to httplib.py
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24674
diff changeset
384 if tr_enc and tr_enc.lower() == "chunked":
50540
4229a4b14f1d Merged revisions 67653,67655,67682-67683,67724,67755,67780-67783,67786,67789,67841,67843,67865,67881,67884,67925,67929,67931 via svnmerge from
Georg Brandl <georg@python.org>
parents: 49907
diff changeset
385 self.chunked = True
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
386 self.chunk_left = None
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
387 else:
50540
4229a4b14f1d Merged revisions 67653,67655,67682-67683,67724,67755,67780-67783,67786,67789,67841,67843,67865,67881,67884,67925,67929,67931 via svnmerge from
Georg Brandl <georg@python.org>
parents: 49907
diff changeset
388 self.chunked = False
1555
6d35308261f4 Towards a standard access mechanism for URLs.
Guido van Rossum <guido@python.org>
parents:
diff changeset
389
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
390 # will the connection close at the end of the response?
26187
ae169f5b94e0 Fix SF bug #637789: Handle Proxy-Connection header.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 26126
diff changeset
391 self.will_close = self._check_close()
11527
28b8585354de OpenSSL support. This is based on patches for a version of SSLeay by
Guido van Rossum <guido@python.org>
parents: 7122
diff changeset
392
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
393 # do we have a Content-Length?
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
394 # NOTE: RFC 2616, S4.4, #3 says we ignore this if tr_enc is "chunked"
42442
0fa84ffba769 Fix several more paths from the SSL code.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 42440
diff changeset
395 self.length = None
52522
9eceb618274a Simplify the Request class. The basic components of the parsed
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52375
diff changeset
396 length = self.headers.get("content-length")
47635
f31e32a2013c Patch for issue 2848, mostly by Humberto Diogenes, with a couple of
Barry Warsaw <barry@python.org>
parents: 47559
diff changeset
397
f31e32a2013c Patch for issue 2848, mostly by Humberto Diogenes, with a couple of
Barry Warsaw <barry@python.org>
parents: 47559
diff changeset
398 # are we using the chunked-style of transfer encoding?
52522
9eceb618274a Simplify the Request class. The basic components of the parsed
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52375
diff changeset
399 tr_enc = self.headers.get("transfer-encoding")
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
400 if length and not self.chunked:
14560
eeb8ee488106 cope with weird Content-Length values returned from servers by
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14140
diff changeset
401 try:
eeb8ee488106 cope with weird Content-Length values returned from servers by
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14140
diff changeset
402 self.length = int(length)
eeb8ee488106 cope with weird Content-Length values returned from servers by
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14140
diff changeset
403 except ValueError:
45504
1b6fb5699634 Merged revisions 61034-61036,61038-61048 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents: 45471
diff changeset
404 self.length = None
1b6fb5699634 Merged revisions 61034-61036,61038-61048 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents: 45471
diff changeset
405 else:
1b6fb5699634 Merged revisions 61034-61036,61038-61048 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents: 45471
diff changeset
406 if self.length < 0: # ignore nonsensical negative lengths
1b6fb5699634 Merged revisions 61034-61036,61038-61048 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents: 45471
diff changeset
407 self.length = None
1b6fb5699634 Merged revisions 61034-61036,61038-61048 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents: 45471
diff changeset
408 else:
1b6fb5699634 Merged revisions 61034-61036,61038-61048 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents: 45471
diff changeset
409 self.length = None
11527
28b8585354de OpenSSL support. This is based on patches for a version of SSLeay by
Guido van Rossum <guido@python.org>
parents: 7122
diff changeset
410
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
411 # does the body have a fixed length? (of zero)
33506
4b8198a74873 Patch #1025790: Add status code constants to httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 33496
diff changeset
412 if (status == NO_CONTENT or status == NOT_MODIFIED or
28978
83fe0f5c9359 SF bug 622042: Don't expect response body from HEAD request.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 28671
diff changeset
413 100 <= status < 200 or # 1xx codes
42426
ae72bf5ae465 Make consistent use of "" for string literals in new classes.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 42425
diff changeset
414 self._method == "HEAD"):
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
415 self.length = 0
1555
6d35308261f4 Towards a standard access mechanism for URLs.
Guido van Rossum <guido@python.org>
parents:
diff changeset
416
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
417 # if the connection remains open, and we aren't using chunked, and
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
418 # a content-length was not provided, then assume that the connection
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
419 # WILL close.
42442
0fa84ffba769 Fix several more paths from the SSL code.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 42440
diff changeset
420 if (not self.will_close and
0fa84ffba769 Fix several more paths from the SSL code.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 42440
diff changeset
421 not self.chunked and
0fa84ffba769 Fix several more paths from the SSL code.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 42440
diff changeset
422 self.length is None):
50540
4229a4b14f1d Merged revisions 67653,67655,67682-67683,67724,67755,67780-67783,67786,67789,67841,67843,67865,67881,67884,67925,67929,67931 via svnmerge from
Georg Brandl <georg@python.org>
parents: 49907
diff changeset
423 self.will_close = True
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
424
26187
ae169f5b94e0 Fix SF bug #637789: Handle Proxy-Connection header.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 26126
diff changeset
425 def _check_close(self):
52522
9eceb618274a Simplify the Request class. The basic components of the parsed
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52375
diff changeset
426 conn = self.headers.get("connection")
26187
ae169f5b94e0 Fix SF bug #637789: Handle Proxy-Connection header.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 26126
diff changeset
427 if self.version == 11:
ae169f5b94e0 Fix SF bug #637789: Handle Proxy-Connection header.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 26126
diff changeset
428 # An HTTP/1.1 proxy is assumed to stay open unless
ae169f5b94e0 Fix SF bug #637789: Handle Proxy-Connection header.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 26126
diff changeset
429 # explicitly closed.
52522
9eceb618274a Simplify the Request class. The basic components of the parsed
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52375
diff changeset
430 conn = self.headers.get("connection")
31967
3d046d063a84 Replace str.find()!=1 with the more readable "in" operator.
Raymond Hettinger <python@rcn.com>
parents: 31389
diff changeset
431 if conn and "close" in conn.lower():
26187
ae169f5b94e0 Fix SF bug #637789: Handle Proxy-Connection header.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 26126
diff changeset
432 return True
ae169f5b94e0 Fix SF bug #637789: Handle Proxy-Connection header.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 26126
diff changeset
433 return False
ae169f5b94e0 Fix SF bug #637789: Handle Proxy-Connection header.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 26126
diff changeset
434
33001
4fd59651fe3f SF bug 874842 and patch 997626: httplib bugs
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 32252
diff changeset
435 # Some HTTP/1.0 implementations have support for persistent
4fd59651fe3f SF bug 874842 and patch 997626: httplib bugs
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 32252
diff changeset
436 # connections, using rules different than HTTP/1.1.
26187
ae169f5b94e0 Fix SF bug #637789: Handle Proxy-Connection header.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 26126
diff changeset
437
44230
6b517ac10e8e Merged revisions 59407-59422 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents: 43777
diff changeset
438 # For older HTTP, Keep-Alive indicates persistent connection.
52522
9eceb618274a Simplify the Request class. The basic components of the parsed
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52375
diff changeset
439 if self.headers.get("keep-alive"):
26187
ae169f5b94e0 Fix SF bug #637789: Handle Proxy-Connection header.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 26126
diff changeset
440 return False
26387
fcf9734e1021 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 26188
diff changeset
441
33001
4fd59651fe3f SF bug 874842 and patch 997626: httplib bugs
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 32252
diff changeset
442 # At least Akamai returns a "Connection: Keep-Alive" header,
4fd59651fe3f SF bug 874842 and patch 997626: httplib bugs
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 32252
diff changeset
443 # which was supposed to be sent by the client.
4fd59651fe3f SF bug 874842 and patch 997626: httplib bugs
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 32252
diff changeset
444 if conn and "keep-alive" in conn.lower():
4fd59651fe3f SF bug 874842 and patch 997626: httplib bugs
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 32252
diff changeset
445 return False
4fd59651fe3f SF bug 874842 and patch 997626: httplib bugs
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 32252
diff changeset
446
26187
ae169f5b94e0 Fix SF bug #637789: Handle Proxy-Connection header.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 26126
diff changeset
447 # Proxy-Connection is a netscape hack.
52522
9eceb618274a Simplify the Request class. The basic components of the parsed
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52375
diff changeset
448 pconn = self.headers.get("proxy-connection")
31967
3d046d063a84 Replace str.find()!=1 with the more readable "in" operator.
Raymond Hettinger <python@rcn.com>
parents: 31389
diff changeset
449 if pconn and "keep-alive" in pconn.lower():
26187
ae169f5b94e0 Fix SF bug #637789: Handle Proxy-Connection header.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 26126
diff changeset
450 return False
ae169f5b94e0 Fix SF bug #637789: Handle Proxy-Connection header.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 26126
diff changeset
451
ae169f5b94e0 Fix SF bug #637789: Handle Proxy-Connection header.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 26126
diff changeset
452 # otherwise, assume it will close
ae169f5b94e0 Fix SF bug #637789: Handle Proxy-Connection header.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 26126
diff changeset
453 return True
ae169f5b94e0 Fix SF bug #637789: Handle Proxy-Connection header.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 26126
diff changeset
454
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
455 def close(self):
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
456 if self.fp:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
457 self.fp.close()
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
458 self.fp = None
11527
28b8585354de OpenSSL support. This is based on patches for a version of SSLeay by
Guido van Rossum <guido@python.org>
parents: 7122
diff changeset
459
42497
17cdbedbb15d Fix several failing tests in test_urllib2net.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 42466
diff changeset
460 # These implementations are for the benefit of io.BufferedReader.
17cdbedbb15d Fix several failing tests in test_urllib2net.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 42466
diff changeset
461
17cdbedbb15d Fix several failing tests in test_urllib2net.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 42466
diff changeset
462 # XXX This class should probably be revised to act more like
17cdbedbb15d Fix several failing tests in test_urllib2net.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 42466
diff changeset
463 # the "raw stream" that BufferedReader expects.
17cdbedbb15d Fix several failing tests in test_urllib2net.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 42466
diff changeset
464
17cdbedbb15d Fix several failing tests in test_urllib2net.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 42466
diff changeset
465 @property
17cdbedbb15d Fix several failing tests in test_urllib2net.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 42466
diff changeset
466 def closed(self):
17cdbedbb15d Fix several failing tests in test_urllib2net.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 42466
diff changeset
467 return self.isclosed()
17cdbedbb15d Fix several failing tests in test_urllib2net.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 42466
diff changeset
468
17cdbedbb15d Fix several failing tests in test_urllib2net.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 42466
diff changeset
469 def flush(self):
17cdbedbb15d Fix several failing tests in test_urllib2net.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 42466
diff changeset
470 self.fp.flush()
17cdbedbb15d Fix several failing tests in test_urllib2net.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 42466
diff changeset
471
52631
d85090a480ba An HTTPResponse is, by its nature, readable.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52522
diff changeset
472 def readable(self):
d85090a480ba An HTTPResponse is, by its nature, readable.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52522
diff changeset
473 return True
d85090a480ba An HTTPResponse is, by its nature, readable.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52522
diff changeset
474
42497
17cdbedbb15d Fix several failing tests in test_urllib2net.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 42466
diff changeset
475 # End of "raw stream" methods
17cdbedbb15d Fix several failing tests in test_urllib2net.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 42466
diff changeset
476
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
477 def isclosed(self):
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
478 # NOTE: it is possible that we will not ever call self.close(). This
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
479 # case occurs when will_close is TRUE, length is None, and we
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
480 # read up to the last byte, but NOT past it.
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
481 #
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
482 # IMPLIES: if will_close is FALSE, then self.close() will ALWAYS be
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
483 # called, meaning self.isclosed() is meaningful.
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
484 return self.fp is None
6787
f6954274e1ca Doc strings by Mitch Chapman (with a little reformatting).
Guido van Rossum <guido@python.org>
parents: 5974
diff changeset
485
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
486 def read(self, amt=None):
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
487 if self.fp is None:
43734
3e5edf19d5f0 Merged revisions 58221-58741 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 43400
diff changeset
488 return b""
1555
6d35308261f4 Towards a standard access mechanism for URLs.
Guido van Rossum <guido@python.org>
parents:
diff changeset
489
61060
fd7ec607e101 Merged revisions 80583 via svnmerge from
Senthil Kumaran <orsenthil@gmail.com>
parents: 59875
diff changeset
490 if self._method == "HEAD":
62066
077d29384399 Merged revisions 81687 via svnmerge from
Senthil Kumaran <orsenthil@gmail.com>
parents: 61060
diff changeset
491 self.close()
61060
fd7ec607e101 Merged revisions 80583 via svnmerge from
Senthil Kumaran <orsenthil@gmail.com>
parents: 59875
diff changeset
492 return b""
fd7ec607e101 Merged revisions 80583 via svnmerge from
Senthil Kumaran <orsenthil@gmail.com>
parents: 59875
diff changeset
493
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
494 if self.chunked:
25112
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
495 return self._read_chunked(amt)
26126
98f3fd1ed42f Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 25112
diff changeset
496
25112
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
497 if amt is None:
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
498 # unbounded read
33933
eb231544c1ad Fix for SF bug 988120 via patch 1061941.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 33761
diff changeset
499 if self.length is None:
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
500 s = self.fp.read()
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
501 else:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
502 s = self._safe_read(self.length)
33933
eb231544c1ad Fix for SF bug 988120 via patch 1061941.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 33761
diff changeset
503 self.length = 0
15868
3060cbfa01be Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15859
diff changeset
504 self.close() # we read everything
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
505 return s
1555
6d35308261f4 Towards a standard access mechanism for URLs.
Guido van Rossum <guido@python.org>
parents:
diff changeset
506
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
507 if self.length is not None:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
508 if amt > self.length:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
509 # clip the read to the "end of response"
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
510 amt = self.length
1555
6d35308261f4 Towards a standard access mechanism for URLs.
Guido van Rossum <guido@python.org>
parents:
diff changeset
511
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
512 # we do not use _safe_read() here because this may be a .will_close
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
513 # connection, and the user is reading more bytes than will be provided
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
514 # (for example, reading in 1k chunks)
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
515 s = self.fp.read(amt)
33933
eb231544c1ad Fix for SF bug 988120 via patch 1061941.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 33761
diff changeset
516 if self.length is not None:
eb231544c1ad Fix for SF bug 988120 via patch 1061941.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 33761
diff changeset
517 self.length -= len(s)
43734
3e5edf19d5f0 Merged revisions 58221-58741 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 43400
diff changeset
518 if not self.length:
3e5edf19d5f0 Merged revisions 58221-58741 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 43400
diff changeset
519 self.close()
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
520 return s
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
521
25112
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
522 def _read_chunked(self, amt):
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
523 assert self.chunked != _UNKNOWN
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
524 chunk_left = self.chunk_left
55830
d567d963cb37 Fixes issue #6838: use a list to accumulate the value instead of repeatedly concatenating strings.
Chris Withers <chris@simplistix.co.uk>
parents: 55438
diff changeset
525 value = []
28083
29965d0d4650 Module review:
Raymond Hettinger <python@rcn.com>
parents: 28063
diff changeset
526 while True:
25112
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
527 if chunk_left is None:
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
528 line = self.fp.readline()
44967
221dec03e261 #1929: fix httplib _read_chunked (str/bytes confusion).
Georg Brandl <georg@python.org>
parents: 44230
diff changeset
529 i = line.find(b";")
25112
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
530 if i >= 0:
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
531 line = line[:i] # strip chunk-extensions
45504
1b6fb5699634 Merged revisions 61034-61036,61038-61048 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents: 45471
diff changeset
532 try:
1b6fb5699634 Merged revisions 61034-61036,61038-61048 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents: 45471
diff changeset
533 chunk_left = int(line, 16)
1b6fb5699634 Merged revisions 61034-61036,61038-61048 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents: 45471
diff changeset
534 except ValueError:
1b6fb5699634 Merged revisions 61034-61036,61038-61048 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents: 45471
diff changeset
535 # close the connection as protocol synchronisation is
1b6fb5699634 Merged revisions 61034-61036,61038-61048 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents: 45471
diff changeset
536 # probably lost
1b6fb5699634 Merged revisions 61034-61036,61038-61048 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents: 45471
diff changeset
537 self.close()
55830
d567d963cb37 Fixes issue #6838: use a list to accumulate the value instead of repeatedly concatenating strings.
Chris Withers <chris@simplistix.co.uk>
parents: 55438
diff changeset
538 raise IncompleteRead(b''.join(value))
25112
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
539 if chunk_left == 0:
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
540 break
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
541 if amt is None:
55830
d567d963cb37 Fixes issue #6838: use a list to accumulate the value instead of repeatedly concatenating strings.
Chris Withers <chris@simplistix.co.uk>
parents: 55438
diff changeset
542 value.append(self._safe_read(chunk_left))
25112
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
543 elif amt < chunk_left:
55830
d567d963cb37 Fixes issue #6838: use a list to accumulate the value instead of repeatedly concatenating strings.
Chris Withers <chris@simplistix.co.uk>
parents: 55438
diff changeset
544 value.append(self._safe_read(amt))
25112
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
545 self.chunk_left = chunk_left - amt
55830
d567d963cb37 Fixes issue #6838: use a list to accumulate the value instead of repeatedly concatenating strings.
Chris Withers <chris@simplistix.co.uk>
parents: 55438
diff changeset
546 return b''.join(value)
25112
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
547 elif amt == chunk_left:
55830
d567d963cb37 Fixes issue #6838: use a list to accumulate the value instead of repeatedly concatenating strings.
Chris Withers <chris@simplistix.co.uk>
parents: 55438
diff changeset
548 value.append(self._safe_read(amt))
25112
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
549 self._safe_read(2) # toss the CRLF at the end of the chunk
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
550 self.chunk_left = None
55830
d567d963cb37 Fixes issue #6838: use a list to accumulate the value instead of repeatedly concatenating strings.
Chris Withers <chris@simplistix.co.uk>
parents: 55438
diff changeset
551 return b''.join(value)
25112
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
552 else:
55830
d567d963cb37 Fixes issue #6838: use a list to accumulate the value instead of repeatedly concatenating strings.
Chris Withers <chris@simplistix.co.uk>
parents: 55438
diff changeset
553 value.append(self._safe_read(chunk_left))
25112
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
554 amt -= chunk_left
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
555
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
556 # we read the whole chunk, get another
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
557 self._safe_read(2) # toss the CRLF at the end of the chunk
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
558 chunk_left = None
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
559
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
560 # read and discard trailer up to the CRLF terminator
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
561 ### note: we shouldn't have any trailers!
28083
29965d0d4650 Module review:
Raymond Hettinger <python@rcn.com>
parents: 28063
diff changeset
562 while True:
25112
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
563 line = self.fp.readline()
45307
f1205121e848 Merged revisions 60481,60485,60489-60492,60494-60496,60498-60499,60501-60503,60505-60506,60508-60509,60523-60524,60532,60543,60545,60547-60548,60552,60554,60556-60559,60561-60562,60569,60571-60572,60574,60576-60583,60585-60586,60589,60591,60594-60595,60597-60598,60600-60601,60606-60612,60615,60617,60619-60621,60623-60625,60627-60629,60631,60633,60635,60647,60650,60652,60654,60656,60658-60659,60664-60666,60668-60670,60672,60676,60678,60680-60683,60685-60686,60688,60690,60692-60694,60697-60700,60705-60706,60708,60711,60714,60720,60724-60730,60732,60735-60751 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents: 44967
diff changeset
564 if not line:
f1205121e848 Merged revisions 60481,60485,60489-60492,60494-60496,60498-60499,60501-60503,60505-60506,60508-60509,60523-60524,60532,60543,60545,60547-60548,60552,60554,60556-60559,60561-60562,60569,60571-60572,60574,60576-60583,60585-60586,60589,60591,60594-60595,60597-60598,60600-60601,60606-60612,60615,60617,60619-60621,60623-60625,60627-60629,60631,60633,60635,60647,60650,60652,60654,60656,60658-60659,60664-60666,60668-60670,60672,60676,60678,60680-60683,60685-60686,60688,60690,60692-60694,60697-60700,60705-60706,60708,60711,60714,60720,60724-60730,60732,60735-60751 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents: 44967
diff changeset
565 # a vanishingly small number of sites EOF without
f1205121e848 Merged revisions 60481,60485,60489-60492,60494-60496,60498-60499,60501-60503,60505-60506,60508-60509,60523-60524,60532,60543,60545,60547-60548,60552,60554,60556-60559,60561-60562,60569,60571-60572,60574,60576-60583,60585-60586,60589,60591,60594-60595,60597-60598,60600-60601,60606-60612,60615,60617,60619-60621,60623-60625,60627-60629,60631,60633,60635,60647,60650,60652,60654,60656,60658-60659,60664-60666,60668-60670,60672,60676,60678,60680-60683,60685-60686,60688,60690,60692-60694,60697-60700,60705-60706,60708,60711,60714,60720,60724-60730,60732,60735-60751 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents: 44967
diff changeset
566 # sending the trailer
f1205121e848 Merged revisions 60481,60485,60489-60492,60494-60496,60498-60499,60501-60503,60505-60506,60508-60509,60523-60524,60532,60543,60545,60547-60548,60552,60554,60556-60559,60561-60562,60569,60571-60572,60574,60576-60583,60585-60586,60589,60591,60594-60595,60597-60598,60600-60601,60606-60612,60615,60617,60619-60621,60623-60625,60627-60629,60631,60633,60635,60647,60650,60652,60654,60656,60658-60659,60664-60666,60668-60670,60672,60676,60678,60680-60683,60685-60686,60688,60690,60692-60694,60697-60700,60705-60706,60708,60711,60714,60720,60724-60730,60732,60735-60751 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents: 44967
diff changeset
567 break
43734
3e5edf19d5f0 Merged revisions 58221-58741 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 43400
diff changeset
568 if line == b"\r\n":
25112
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
569 break
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
570
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
571 # we read everything; close the "file"
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
572 self.close()
f0ff060cd194 Move code for reading chunked responses in helper function,
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 25107
diff changeset
573
55830
d567d963cb37 Fixes issue #6838: use a list to accumulate the value instead of repeatedly concatenating strings.
Chris Withers <chris@simplistix.co.uk>
parents: 55438
diff changeset
574 return b''.join(value)
26126
98f3fd1ed42f Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 25112
diff changeset
575
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
576 def _safe_read(self, amt):
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
577 """Read the number of bytes requested, compensating for partial reads.
11527
28b8585354de OpenSSL support. This is based on patches for a version of SSLeay by
Guido van Rossum <guido@python.org>
parents: 7122
diff changeset
578
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
579 Normally, we have a blocking socket, but a read() can be interrupted
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
580 by a signal (resulting in a partial read).
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
581
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
582 Note that we cannot distinguish between EOF and an interrupt when zero
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
583 bytes have been read. IncompleteRead() will be raised in this
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
584 situation.
11527
28b8585354de OpenSSL support. This is based on patches for a version of SSLeay by
Guido van Rossum <guido@python.org>
parents: 7122
diff changeset
585
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
586 This function should be used when <amt> bytes "should" be present for
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
587 reading. If the bytes are truly not available (due to EOF), then the
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
588 IncompleteRead exception can be used to detect the problem.
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
589 """
35652
6a20831d357d bug [ 1296004 ] MemoryError in httplib
Georg Brandl <georg@python.org>
parents: 35158
diff changeset
590 s = []
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
591 while amt > 0:
35652
6a20831d357d bug [ 1296004 ] MemoryError in httplib
Georg Brandl <georg@python.org>
parents: 35158
diff changeset
592 chunk = self.fp.read(min(amt, MAXAMOUNT))
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
593 if not chunk:
51953
9c56eb17f334 Merged revisions 70107 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 51532
diff changeset
594 raise IncompleteRead(b''.join(s), amt)
35652
6a20831d357d bug [ 1296004 ] MemoryError in httplib
Georg Brandl <georg@python.org>
parents: 35158
diff changeset
595 s.append(chunk)
28083
29965d0d4650 Module review:
Raymond Hettinger <python@rcn.com>
parents: 28063
diff changeset
596 amt -= len(chunk)
43351
84dc86f21196 Fix for bug 1148: str/bytes issue in httplib's _safe_read().
Guido van Rossum <guido@python.org>
parents: 43292
diff changeset
597 return b"".join(s)
11527
28b8585354de OpenSSL support. This is based on patches for a version of SSLeay by
Guido van Rossum <guido@python.org>
parents: 7122
diff changeset
598
51532
af469dc15fa7 Issue #4631: Fix urlopen() result when an HTTP response uses chunked encoding.
Antoine Pitrou <solipsis@pitrou.net>
parents: 51449
diff changeset
599 def fileno(self):
af469dc15fa7 Issue #4631: Fix urlopen() result when an HTTP response uses chunked encoding.
Antoine Pitrou <solipsis@pitrou.net>
parents: 51449
diff changeset
600 return self.fp.fileno()
af469dc15fa7 Issue #4631: Fix urlopen() result when an HTTP response uses chunked encoding.
Antoine Pitrou <solipsis@pitrou.net>
parents: 51449
diff changeset
601
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
602 def getheader(self, name, default=None):
52522
9eceb618274a Simplify the Request class. The basic components of the parsed
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52375
diff changeset
603 if self.headers is None:
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
604 raise ResponseNotReady()
63537
39e14df765e4 Fix Issue8572 - httplib getheader() throws error instead of default
Senthil Kumaran <orsenthil@gmail.com>
parents: 63192
diff changeset
605 headers = self.headers.get_all(name) or default
39e14df765e4 Fix Issue8572 - httplib getheader() throws error instead of default
Senthil Kumaran <orsenthil@gmail.com>
parents: 63192
diff changeset
606 if isinstance(headers, str) or not hasattr(headers, '__iter__'):
39e14df765e4 Fix Issue8572 - httplib getheader() throws error instead of default
Senthil Kumaran <orsenthil@gmail.com>
parents: 63192
diff changeset
607 return headers
39e14df765e4 Fix Issue8572 - httplib getheader() throws error instead of default
Senthil Kumaran <orsenthil@gmail.com>
parents: 63192
diff changeset
608 else:
39e14df765e4 Fix Issue8572 - httplib getheader() throws error instead of default
Senthil Kumaran <orsenthil@gmail.com>
parents: 63192
diff changeset
609 return ', '.join(headers)
11527
28b8585354de OpenSSL support. This is based on patches for a version of SSLeay by
Guido van Rossum <guido@python.org>
parents: 7122
diff changeset
610
33173
e05e9d05830d Patch #800236: add HTTPResponse.getheaders().
Martin v. Löwis <martin@v.loewis.de>
parents: 33001
diff changeset
611 def getheaders(self):
e05e9d05830d Patch #800236: add HTTPResponse.getheaders().
Martin v. Löwis <martin@v.loewis.de>
parents: 33001
diff changeset
612 """Return list of (header, value) tuples."""
52522
9eceb618274a Simplify the Request class. The basic components of the parsed
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52375
diff changeset
613 if self.headers is None:
33173
e05e9d05830d Patch #800236: add HTTPResponse.getheaders().
Martin v. Löwis <martin@v.loewis.de>
parents: 33001
diff changeset
614 raise ResponseNotReady()
52522
9eceb618274a Simplify the Request class. The basic components of the parsed
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52375
diff changeset
615 return list(self.headers.items())
33173
e05e9d05830d Patch #800236: add HTTPResponse.getheaders().
Martin v. Löwis <martin@v.loewis.de>
parents: 33001
diff changeset
616
51532
af469dc15fa7 Issue #4631: Fix urlopen() result when an HTTP response uses chunked encoding.
Antoine Pitrou <solipsis@pitrou.net>
parents: 51449
diff changeset
617 # We override IOBase.__iter__ so that it doesn't check for closed-ness
af469dc15fa7 Issue #4631: Fix urlopen() result when an HTTP response uses chunked encoding.
Antoine Pitrou <solipsis@pitrou.net>
parents: 51449
diff changeset
618
af469dc15fa7 Issue #4631: Fix urlopen() result when an HTTP response uses chunked encoding.
Antoine Pitrou <solipsis@pitrou.net>
parents: 51449
diff changeset
619 def __iter__(self):
af469dc15fa7 Issue #4631: Fix urlopen() result when an HTTP response uses chunked encoding.
Antoine Pitrou <solipsis@pitrou.net>
parents: 51449
diff changeset
620 return self
af469dc15fa7 Issue #4631: Fix urlopen() result when an HTTP response uses chunked encoding.
Antoine Pitrou <solipsis@pitrou.net>
parents: 51449
diff changeset
621
52522
9eceb618274a Simplify the Request class. The basic components of the parsed
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52375
diff changeset
622 # For compatibility with old-style urllib responses.
9eceb618274a Simplify the Request class. The basic components of the parsed
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52375
diff changeset
623
9eceb618274a Simplify the Request class. The basic components of the parsed
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52375
diff changeset
624 def info(self):
9eceb618274a Simplify the Request class. The basic components of the parsed
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52375
diff changeset
625 return self.headers
9eceb618274a Simplify the Request class. The basic components of the parsed
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52375
diff changeset
626
9eceb618274a Simplify the Request class. The basic components of the parsed
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52375
diff changeset
627 def geturl(self):
9eceb618274a Simplify the Request class. The basic components of the parsed
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52375
diff changeset
628 return self.url
9eceb618274a Simplify the Request class. The basic components of the parsed
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52375
diff changeset
629
9eceb618274a Simplify the Request class. The basic components of the parsed
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52375
diff changeset
630 def getcode(self):
9eceb618274a Simplify the Request class. The basic components of the parsed
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52375
diff changeset
631 return self.status
1555
6d35308261f4 Towards a standard access mechanism for URLs.
Guido van Rossum <guido@python.org>
parents:
diff changeset
632
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
633 class HTTPConnection:
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
634
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
635 _http_vsn = 11
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
636 _http_vsn_str = 'HTTP/1.1'
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
637
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
638 response_class = HTTPResponse
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
639 default_port = HTTP_PORT
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
640 auto_open = 1
14672
1b20ad6eb501 Do not close socket when a Content-Length is 0. This make the
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14560
diff changeset
641 debuglevel = 0
24119
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
642 strict = 0
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
643
47559
2925b568aacc Merged revisions 63562,63570,63728,63734,63784,63788,63802,63817,63827,63839,63887,63975,63998 via svnmerge from
Georg Brandl <georg@python.org>
parents: 47318
diff changeset
644 def __init__(self, host, port=None, strict=None,
58114
b0386fe650c7 Merged revisions 77263-77264 via svnmerge from
Gregory P. Smith <greg@mad-scientist.com>
parents: 57801
diff changeset
645 timeout=socket._GLOBAL_DEFAULT_TIMEOUT, source_address=None):
41554
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 41501
diff changeset
646 self.timeout = timeout
58114
b0386fe650c7 Merged revisions 77263-77264 via svnmerge from
Gregory P. Smith <greg@mad-scientist.com>
parents: 57801
diff changeset
647 self.source_address = source_address
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
648 self.sock = None
24262
3b571e32a6e4 Send HTTP requests with a single send() call instead of many.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24216
diff changeset
649 self._buffer = []
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
650 self.__response = None
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
651 self.__state = _CS_IDLE
28978
83fe0f5c9359 SF bug 622042: Don't expect response body from HEAD request.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 28671
diff changeset
652 self._method = None
55438
d1ec95ae0ac2 Fixed Issue1424152 in Py3k: urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 52631
diff changeset
653 self._tunnel_host = None
d1ec95ae0ac2 Fixed Issue1424152 in Py3k: urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 52631
diff changeset
654 self._tunnel_port = None
24263
e6033254df21 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 24262
diff changeset
655
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
656 self._set_hostport(host, port)
24119
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
657 if strict is not None:
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
658 self.strict = strict
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
659
57801
724a0da85c42 Merged revisions 76908 via svnmerge from
Senthil Kumaran <orsenthil@gmail.com>
parents: 56288
diff changeset
660 def set_tunnel(self, host, port=None, headers=None):
724a0da85c42 Merged revisions 76908 via svnmerge from
Senthil Kumaran <orsenthil@gmail.com>
parents: 56288
diff changeset
661 """ Sets up the host and the port for the HTTP CONNECT Tunnelling.
724a0da85c42 Merged revisions 76908 via svnmerge from
Senthil Kumaran <orsenthil@gmail.com>
parents: 56288
diff changeset
662
724a0da85c42 Merged revisions 76908 via svnmerge from
Senthil Kumaran <orsenthil@gmail.com>
parents: 56288
diff changeset
663 The headers argument should be a mapping of extra HTTP headers
724a0da85c42 Merged revisions 76908 via svnmerge from
Senthil Kumaran <orsenthil@gmail.com>
parents: 56288
diff changeset
664 to send with the CONNECT request.
724a0da85c42 Merged revisions 76908 via svnmerge from
Senthil Kumaran <orsenthil@gmail.com>
parents: 56288
diff changeset
665 """
55438
d1ec95ae0ac2 Fixed Issue1424152 in Py3k: urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 52631
diff changeset
666 self._tunnel_host = host
d1ec95ae0ac2 Fixed Issue1424152 in Py3k: urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 52631
diff changeset
667 self._tunnel_port = port
57801
724a0da85c42 Merged revisions 76908 via svnmerge from
Senthil Kumaran <orsenthil@gmail.com>
parents: 56288
diff changeset
668 if headers:
724a0da85c42 Merged revisions 76908 via svnmerge from
Senthil Kumaran <orsenthil@gmail.com>
parents: 56288
diff changeset
669 self._tunnel_headers = headers
724a0da85c42 Merged revisions 76908 via svnmerge from
Senthil Kumaran <orsenthil@gmail.com>
parents: 56288
diff changeset
670 else:
724a0da85c42 Merged revisions 76908 via svnmerge from
Senthil Kumaran <orsenthil@gmail.com>
parents: 56288
diff changeset
671 self._tunnel_headers.clear()
55438
d1ec95ae0ac2 Fixed Issue1424152 in Py3k: urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 52631
diff changeset
672
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
673 def _set_hostport(self, host, port):
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
674 if port is None:
33488
71ed4b35fdc4 Search from the end of the host/port combination to find the colon which
Skip Montanaro <skip@pobox.com>
parents: 33173
diff changeset
675 i = host.rfind(':')
33491
0c299cabf4f0 missed the obvious test case and corresponding fix
Skip Montanaro <skip@pobox.com>
parents: 33488
diff changeset
676 j = host.rfind(']') # ipv6 addresses have [...]
0c299cabf4f0 missed the obvious test case and corresponding fix
Skip Montanaro <skip@pobox.com>
parents: 33488
diff changeset
677 if i > j:
22494
7224a1df4809 add InvalidURL exception - raised if port is given but empty or non-numeric
Skip Montanaro <skip@pobox.com>
parents: 22403
diff changeset
678 try:
7224a1df4809 add InvalidURL exception - raised if port is given but empty or non-numeric
Skip Montanaro <skip@pobox.com>
parents: 22403
diff changeset
679 port = int(host[i+1:])
7224a1df4809 add InvalidURL exception - raised if port is given but empty or non-numeric
Skip Montanaro <skip@pobox.com>
parents: 22403
diff changeset
680 except ValueError:
24088
b74d4a59793a Convert raise to call exception class. Add whitespace.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24042
diff changeset
681 raise InvalidURL("nonnumeric port: '%s'" % host[i+1:])
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
682 host = host[:i]
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
683 else:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
684 port = self.default_port
33761
d1228127d294 SF bug #1046855: httplib index out of range
Raymond Hettinger <python@rcn.com>
parents: 33506
diff changeset
685 if host and host[0] == '[' and host[-1] == ']':
33496
9e2b94a3b5dc Remove tabs.
Brett Cannon <bcannon@gmail.com>
parents: 33493
diff changeset
686 host = host[1:-1]
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
687 self.host = host
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
688 self.port = port
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
689
14672
1b20ad6eb501 Do not close socket when a Content-Length is 0. This make the
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14560
diff changeset
690 def set_debuglevel(self, level):
1b20ad6eb501 Do not close socket when a Content-Length is 0. This make the
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14560
diff changeset
691 self.debuglevel = level
1b20ad6eb501 Do not close socket when a Content-Length is 0. This make the
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14560
diff changeset
692
55438
d1ec95ae0ac2 Fixed Issue1424152 in Py3k: urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 52631
diff changeset
693 def _tunnel(self):
d1ec95ae0ac2 Fixed Issue1424152 in Py3k: urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 52631
diff changeset
694 self._set_hostport(self._tunnel_host, self._tunnel_port)
57801
724a0da85c42 Merged revisions 76908 via svnmerge from
Senthil Kumaran <orsenthil@gmail.com>
parents: 56288
diff changeset
695 connect_str = "CONNECT %s:%d HTTP/1.0\r\n" %(self.host, self.port)
55438
d1ec95ae0ac2 Fixed Issue1424152 in Py3k: urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 52631
diff changeset
696 connect_bytes = connect_str.encode("ascii")
d1ec95ae0ac2 Fixed Issue1424152 in Py3k: urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 52631
diff changeset
697 self.send(connect_bytes)
57801
724a0da85c42 Merged revisions 76908 via svnmerge from
Senthil Kumaran <orsenthil@gmail.com>
parents: 56288
diff changeset
698 for header, value in self._tunnel_headers.iteritems():
724a0da85c42 Merged revisions 76908 via svnmerge from
Senthil Kumaran <orsenthil@gmail.com>
parents: 56288
diff changeset
699 header_str = "%s: %s\r\n" % (header, value)
724a0da85c42 Merged revisions 76908 via svnmerge from
Senthil Kumaran <orsenthil@gmail.com>
parents: 56288
diff changeset
700 header_bytes = header_str.encode("ascii")
724a0da85c42 Merged revisions 76908 via svnmerge from
Senthil Kumaran <orsenthil@gmail.com>
parents: 56288
diff changeset
701 self.send(header_bytes)
724a0da85c42 Merged revisions 76908 via svnmerge from
Senthil Kumaran <orsenthil@gmail.com>
parents: 56288
diff changeset
702
55438
d1ec95ae0ac2 Fixed Issue1424152 in Py3k: urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 52631
diff changeset
703 response = self.response_class(self.sock, strict = self.strict,
57801
724a0da85c42 Merged revisions 76908 via svnmerge from
Senthil Kumaran <orsenthil@gmail.com>
parents: 56288
diff changeset
704 method = self._method)
55438
d1ec95ae0ac2 Fixed Issue1424152 in Py3k: urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 52631
diff changeset
705 (version, code, message) = response._read_status()
57801
724a0da85c42 Merged revisions 76908 via svnmerge from
Senthil Kumaran <orsenthil@gmail.com>
parents: 56288
diff changeset
706
55438
d1ec95ae0ac2 Fixed Issue1424152 in Py3k: urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 52631
diff changeset
707 if code != 200:
d1ec95ae0ac2 Fixed Issue1424152 in Py3k: urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 52631
diff changeset
708 self.close()
d1ec95ae0ac2 Fixed Issue1424152 in Py3k: urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 52631
diff changeset
709 raise socket.error("Tunnel connection failed: %d %s" % (code,
d1ec95ae0ac2 Fixed Issue1424152 in Py3k: urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 52631
diff changeset
710 message.strip()))
d1ec95ae0ac2 Fixed Issue1424152 in Py3k: urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 52631
diff changeset
711 while True:
d1ec95ae0ac2 Fixed Issue1424152 in Py3k: urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 52631
diff changeset
712 line = response.fp.readline()
d1ec95ae0ac2 Fixed Issue1424152 in Py3k: urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 52631
diff changeset
713 if line == b'\r\n':
d1ec95ae0ac2 Fixed Issue1424152 in Py3k: urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 52631
diff changeset
714 break
d1ec95ae0ac2 Fixed Issue1424152 in Py3k: urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 52631
diff changeset
715
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
716 def connect(self):
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
717 """Connect to the host and port specified in __init__."""
41554
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 41501
diff changeset
718 self.sock = socket.create_connection((self.host,self.port),
58114
b0386fe650c7 Merged revisions 77263-77264 via svnmerge from
Gregory P. Smith <greg@mad-scientist.com>
parents: 57801
diff changeset
719 self.timeout, self.source_address)
55438
d1ec95ae0ac2 Fixed Issue1424152 in Py3k: urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 52631
diff changeset
720 if self._tunnel_host:
d1ec95ae0ac2 Fixed Issue1424152 in Py3k: urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 52631
diff changeset
721 self._tunnel()
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
722
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
723 def close(self):
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
724 """Close the connection to the HTTP server."""
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
725 if self.sock:
15868
3060cbfa01be Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 15859
diff changeset
726 self.sock.close() # close it manually... there may be other refs
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
727 self.sock = None
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
728 if self.__response:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
729 self.__response.close()
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
730 self.__response = None
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
731 self.__state = _CS_IDLE
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
732
65052
7c0f1837e8cb Use proper variable name 'data' instead of 'str' in the send method.
Senthil Kumaran <orsenthil@gmail.com>
parents: 63537
diff changeset
733 def send(self, data):
7c0f1837e8cb Use proper variable name 'data' instead of 'str' in the send method.
Senthil Kumaran <orsenthil@gmail.com>
parents: 63537
diff changeset
734 """Send `data' to the server."""
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
735 if self.sock is None:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
736 if self.auto_open:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
737 self.connect()
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
738 else:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
739 raise NotConnected()
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
740
14672
1b20ad6eb501 Do not close socket when a Content-Length is 0. This make the
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14560
diff changeset
741 if self.debuglevel > 0:
65052
7c0f1837e8cb Use proper variable name 'data' instead of 'str' in the send method.
Senthil Kumaran <orsenthil@gmail.com>
parents: 63537
diff changeset
742 print("send:", repr(data))
52375
a726f7928561 Remove special logic that closes HTTPConnection socket on EPIPE.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52370
diff changeset
743 blocksize = 8192
65052
7c0f1837e8cb Use proper variable name 'data' instead of 'str' in the send method.
Senthil Kumaran <orsenthil@gmail.com>
parents: 63537
diff changeset
744 if hasattr(data, "read") :
52375
a726f7928561 Remove special logic that closes HTTPConnection socket on EPIPE.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52370
diff changeset
745 if self.debuglevel > 0:
a726f7928561 Remove special logic that closes HTTPConnection socket on EPIPE.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52370
diff changeset
746 print("sendIng a read()able")
a726f7928561 Remove special logic that closes HTTPConnection socket on EPIPE.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52370
diff changeset
747 encode = False
56288
1f7361c74234 [NOTE: the original bug doesn't exist in py3k but this adds Kirk's tests and fixes
Antoine Pitrou <solipsis@pitrou.net>
parents: 55830
diff changeset
748 try:
65052
7c0f1837e8cb Use proper variable name 'data' instead of 'str' in the send method.
Senthil Kumaran <orsenthil@gmail.com>
parents: 63537
diff changeset
749 mode = data.mode
56288
1f7361c74234 [NOTE: the original bug doesn't exist in py3k but this adds Kirk's tests and fixes
Antoine Pitrou <solipsis@pitrou.net>
parents: 55830
diff changeset
750 except AttributeError:
1f7361c74234 [NOTE: the original bug doesn't exist in py3k but this adds Kirk's tests and fixes
Antoine Pitrou <solipsis@pitrou.net>
parents: 55830
diff changeset
751 # io.BytesIO and other file-like objects don't have a `mode`
1f7361c74234 [NOTE: the original bug doesn't exist in py3k but this adds Kirk's tests and fixes
Antoine Pitrou <solipsis@pitrou.net>
parents: 55830
diff changeset
752 # attribute.
1f7361c74234 [NOTE: the original bug doesn't exist in py3k but this adds Kirk's tests and fixes
Antoine Pitrou <solipsis@pitrou.net>
parents: 55830
diff changeset
753 pass
1f7361c74234 [NOTE: the original bug doesn't exist in py3k but this adds Kirk's tests and fixes
Antoine Pitrou <solipsis@pitrou.net>
parents: 55830
diff changeset
754 else:
1f7361c74234 [NOTE: the original bug doesn't exist in py3k but this adds Kirk's tests and fixes
Antoine Pitrou <solipsis@pitrou.net>
parents: 55830
diff changeset
755 if "b" not in mode:
1f7361c74234 [NOTE: the original bug doesn't exist in py3k but this adds Kirk's tests and fixes
Antoine Pitrou <solipsis@pitrou.net>
parents: 55830
diff changeset
756 encode = True
1f7361c74234 [NOTE: the original bug doesn't exist in py3k but this adds Kirk's tests and fixes
Antoine Pitrou <solipsis@pitrou.net>
parents: 55830
diff changeset
757 if self.debuglevel > 0:
1f7361c74234 [NOTE: the original bug doesn't exist in py3k but this adds Kirk's tests and fixes
Antoine Pitrou <solipsis@pitrou.net>
parents: 55830
diff changeset
758 print("encoding file using iso-8859-1")
52375
a726f7928561 Remove special logic that closes HTTPConnection socket on EPIPE.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52370
diff changeset
759 while 1:
65052
7c0f1837e8cb Use proper variable name 'data' instead of 'str' in the send method.
Senthil Kumaran <orsenthil@gmail.com>
parents: 63537
diff changeset
760 datablock = data.read(blocksize)
7c0f1837e8cb Use proper variable name 'data' instead of 'str' in the send method.
Senthil Kumaran <orsenthil@gmail.com>
parents: 63537
diff changeset
761 if not datablock:
52375
a726f7928561 Remove special logic that closes HTTPConnection socket on EPIPE.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52370
diff changeset
762 break
a726f7928561 Remove special logic that closes HTTPConnection socket on EPIPE.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52370
diff changeset
763 if encode:
65052
7c0f1837e8cb Use proper variable name 'data' instead of 'str' in the send method.
Senthil Kumaran <orsenthil@gmail.com>
parents: 63537
diff changeset
764 datablock = datablock.encode("iso-8859-1")
7c0f1837e8cb Use proper variable name 'data' instead of 'str' in the send method.
Senthil Kumaran <orsenthil@gmail.com>
parents: 63537
diff changeset
765 self.sock.sendall(datablock)
52375
a726f7928561 Remove special logic that closes HTTPConnection socket on EPIPE.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52370
diff changeset
766 else:
65052
7c0f1837e8cb Use proper variable name 'data' instead of 'str' in the send method.
Senthil Kumaran <orsenthil@gmail.com>
parents: 63537
diff changeset
767 self.sock.sendall(data)
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
768
24262
3b571e32a6e4 Send HTTP requests with a single send() call instead of many.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24216
diff changeset
769 def _output(self, s):
3b571e32a6e4 Send HTTP requests with a single send() call instead of many.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24216
diff changeset
770 """Add a line of output to the current request buffer.
24674
3269bdd65988 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 24378
diff changeset
771
24264
18c1b290e484 Fix typos and such caught by the pycheckerbot.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24263
diff changeset
772 Assumes that the line does *not* end with \\r\\n.
24262
3b571e32a6e4 Send HTTP requests with a single send() call instead of many.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24216
diff changeset
773 """
3b571e32a6e4 Send HTTP requests with a single send() call instead of many.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24216
diff changeset
774 self._buffer.append(s)
3b571e32a6e4 Send HTTP requests with a single send() call instead of many.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24216
diff changeset
775
49907
75a8caa24186 Merged revisions 67442 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 49734
diff changeset
776 def _send_output(self, message_body=None):
24262
3b571e32a6e4 Send HTTP requests with a single send() call instead of many.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24216
diff changeset
777 """Send the currently buffered request and clear the buffer.
3b571e32a6e4 Send HTTP requests with a single send() call instead of many.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24216
diff changeset
778
24264
18c1b290e484 Fix typos and such caught by the pycheckerbot.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24263
diff changeset
779 Appends an extra \\r\\n to the buffer.
50842
507874f655ac Merged revisions 68458 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 50540
diff changeset
780 A message_body may be specified, to be appended to the request.
24262
3b571e32a6e4 Send HTTP requests with a single send() call instead of many.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24216
diff changeset
781 """
42133
a8582e25748b Fix test_httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 41824
diff changeset
782 self._buffer.extend((b"", b""))
a8582e25748b Fix test_httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 41824
diff changeset
783 msg = b"\r\n".join(self._buffer)
24262
3b571e32a6e4 Send HTTP requests with a single send() call instead of many.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24216
diff changeset
784 del self._buffer[:]
49907
75a8caa24186 Merged revisions 67442 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 49734
diff changeset
785 # If msg and message_body are sent in a single send() call,
75a8caa24186 Merged revisions 67442 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 49734
diff changeset
786 # it will avoid performance problems caused by the interaction
75a8caa24186 Merged revisions 67442 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 49734
diff changeset
787 # between delayed ack and the Nagle algorithim.
50842
507874f655ac Merged revisions 68458 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 50540
diff changeset
788 if isinstance(message_body, bytes):
49907
75a8caa24186 Merged revisions 67442 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 49734
diff changeset
789 msg += message_body
50842
507874f655ac Merged revisions 68458 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 50540
diff changeset
790 message_body = None
24262
3b571e32a6e4 Send HTTP requests with a single send() call instead of many.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24216
diff changeset
791 self.send(msg)
50842
507874f655ac Merged revisions 68458 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 50540
diff changeset
792 if message_body is not None:
52370
8b691690babd Fix some string encoding issues with entity bodies in HTTP requests.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52369
diff changeset
793 # message_body was not a string (i.e. it is a file), and
8b691690babd Fix some string encoding issues with entity bodies in HTTP requests.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52369
diff changeset
794 # we must run the risk of Nagle.
50842
507874f655ac Merged revisions 68458 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 50540
diff changeset
795 self.send(message_body)
24262
3b571e32a6e4 Send HTTP requests with a single send() call instead of many.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24216
diff changeset
796
30976
ead4c61219d0 Patch #831747: Add skip_accept_encoding parameter to putrequest.
Martin v. Löwis <martin@v.loewis.de>
parents: 30888
diff changeset
797 def putrequest(self, method, url, skip_host=0, skip_accept_encoding=0):
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
798 """Send a request to the server.
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
799
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
800 `method' specifies an HTTP request method, e.g. 'GET'.
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
801 `url' specifies the object being requested, e.g. '/index.html'.
30976
ead4c61219d0 Patch #831747: Add skip_accept_encoding parameter to putrequest.
Martin v. Löwis <martin@v.loewis.de>
parents: 30888
diff changeset
802 `skip_host' if True does not add automatically a 'Host:' header
ead4c61219d0 Patch #831747: Add skip_accept_encoding parameter to putrequest.
Martin v. Löwis <martin@v.loewis.de>
parents: 30888
diff changeset
803 `skip_accept_encoding' if True does not add automatically an
ead4c61219d0 Patch #831747: Add skip_accept_encoding parameter to putrequest.
Martin v. Löwis <martin@v.loewis.de>
parents: 30888
diff changeset
804 'Accept-Encoding:' header
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
805 """
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
806
29584
da0248fcfc52 Deal with a couple XXX comments which asked questions.
Greg Stein <gstein@lyra.org>
parents: 29465
diff changeset
807 # if a prior response has been completed, then forget about it.
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
808 if self.__response and self.__response.isclosed():
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
809 self.__response = None
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
810
31389
aca1ce1b4b61 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 30976
diff changeset
811
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
812 # in certain cases, we cannot issue another request on this connection.
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
813 # this occurs when:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
814 # 1) we are in the process of sending a request. (_CS_REQ_STARTED)
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
815 # 2) a response to a previous request has signalled that it is going
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
816 # to close the connection upon completion.
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
817 # 3) the headers for the previous response have not been read, thus
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
818 # we cannot determine whether point (2) is true. (_CS_REQ_SENT)
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
819 #
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
820 # if there is no prior response, then we can request at will.
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
821 #
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
822 # if point (2) is true, then we will have passed the socket to the
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
823 # response (effectively meaning, "there is no prior response"), and
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
824 # will open a new one when a new request is made.
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
825 #
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
826 # Note: if a prior response exists, then we *can* start a new request.
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
827 # We are not allowed to begin fetching the response to this new
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
828 # request, however, until that prior response is complete.
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
829 #
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
830 if self.__state == _CS_IDLE:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
831 self.__state = _CS_REQ_STARTED
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
832 else:
52522
9eceb618274a Simplify the Request class. The basic components of the parsed
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52375
diff changeset
833 raise CannotSendRequest(self.__state)
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
834
28978
83fe0f5c9359 SF bug 622042: Don't expect response body from HEAD request.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 28671
diff changeset
835 # Save the method we use, we need it later in the response phase
83fe0f5c9359 SF bug 622042: Don't expect response body from HEAD request.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 28671
diff changeset
836 self._method = method
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
837 if not url:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
838 url = '/'
42133
a8582e25748b Fix test_httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 41824
diff changeset
839 request = '%s %s %s' % (method, url, self._http_vsn_str)
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
840
42133
a8582e25748b Fix test_httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 41824
diff changeset
841 # Non-ASCII characters should have been eliminated earlier
a8582e25748b Fix test_httplib.
Martin v. Löwis <martin@v.loewis.de>
parents: 41824
diff changeset
842 self._output(request.encode('ascii'))
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
843
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
844 if self._http_vsn == 11:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
845 # Issue some standard headers for better HTTP/1.1 compliance
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
846
22202
5f2c0f6f0af7 Fix SF bug 525520.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 22192
diff changeset
847 if not skip_host:
5f2c0f6f0af7 Fix SF bug 525520.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 22192
diff changeset
848 # this header is issued *only* for HTTP/1.1
5f2c0f6f0af7 Fix SF bug 525520.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 22192
diff changeset
849 # connections. more specifically, this means it is
5f2c0f6f0af7 Fix SF bug 525520.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 22192
diff changeset
850 # only issued when the client uses the new
5f2c0f6f0af7 Fix SF bug 525520.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 22192
diff changeset
851 # HTTPConnection() class. backwards-compat clients
5f2c0f6f0af7 Fix SF bug 525520.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 22192
diff changeset
852 # will be using HTTP/1.0 and those clients may be
5f2c0f6f0af7 Fix SF bug 525520.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 22192
diff changeset
853 # issuing this header themselves. we should NOT issue
5f2c0f6f0af7 Fix SF bug 525520.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 22192
diff changeset
854 # it twice; some web servers (such as Apache) barf
5f2c0f6f0af7 Fix SF bug 525520.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 22192
diff changeset
855 # when they see two Host: headers
15859
31967a233600 SF Patch #103232 by dougfort: Preserve Nonstandard Port Number in Host
Guido van Rossum <guido@python.org>
parents: 15845
diff changeset
856
22202
5f2c0f6f0af7 Fix SF bug 525520.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 22192
diff changeset
857 # If we need a non-standard port,include it in the
5f2c0f6f0af7 Fix SF bug 525520.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 22192
diff changeset
858 # header. If the request is going through a proxy,
5f2c0f6f0af7 Fix SF bug 525520.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 22192
diff changeset
859 # but the host of the actual URL, not the host of the
5f2c0f6f0af7 Fix SF bug 525520.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 22192
diff changeset
860 # proxy.
22192
044f9e71320f SF bug report #405939: wrong Host header with proxy
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 21962
diff changeset
861
22202
5f2c0f6f0af7 Fix SF bug 525520.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 22192
diff changeset
862 netloc = ''
5f2c0f6f0af7 Fix SF bug 525520.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 22192
diff changeset
863 if url.startswith('http'):
5f2c0f6f0af7 Fix SF bug 525520.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 22192
diff changeset
864 nil, netloc, nil, nil, nil = urlsplit(url)
5f2c0f6f0af7 Fix SF bug 525520.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 22192
diff changeset
865
5f2c0f6f0af7 Fix SF bug 525520.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 22192
diff changeset
866 if netloc:
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 36559
diff changeset
867 try:
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 36559
diff changeset
868 netloc_enc = netloc.encode("ascii")
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 36559
diff changeset
869 except UnicodeEncodeError:
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 36559
diff changeset
870 netloc_enc = netloc.encode("idna")
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 36559
diff changeset
871 self.putheader('Host', netloc_enc)
22202
5f2c0f6f0af7 Fix SF bug 525520.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 22192
diff changeset
872 else:
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 36559
diff changeset
873 try:
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 36559
diff changeset
874 host_enc = self.host.encode("ascii")
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 36559
diff changeset
875 except UnicodeEncodeError:
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 36559
diff changeset
876 host_enc = self.host.encode("idna")
66187
c6b2f417830e Fix Issue5111 - Wrap the Ipv6 host with [] in the Host header
Senthil Kumaran <orsenthil@gmail.com>
parents: 65593
diff changeset
877
c6b2f417830e Fix Issue5111 - Wrap the Ipv6 host with [] in the Host header
Senthil Kumaran <orsenthil@gmail.com>
parents: 65593
diff changeset
878 # As per RFC 273, IPv6 address should be wrapped with []
c6b2f417830e Fix Issue5111 - Wrap the Ipv6 host with [] in the Host header
Senthil Kumaran <orsenthil@gmail.com>
parents: 65593
diff changeset
879 # when used as Host header
c6b2f417830e Fix Issue5111 - Wrap the Ipv6 host with [] in the Host header
Senthil Kumaran <orsenthil@gmail.com>
parents: 65593
diff changeset
880
c6b2f417830e Fix Issue5111 - Wrap the Ipv6 host with [] in the Host header
Senthil Kumaran <orsenthil@gmail.com>
parents: 65593
diff changeset
881 if self.host.find(':') >= 0:
c6b2f417830e Fix Issue5111 - Wrap the Ipv6 host with [] in the Host header
Senthil Kumaran <orsenthil@gmail.com>
parents: 65593
diff changeset
882 host_enc = b'[' + host_enc + b']'
c6b2f417830e Fix Issue5111 - Wrap the Ipv6 host with [] in the Host header
Senthil Kumaran <orsenthil@gmail.com>
parents: 65593
diff changeset
883
48494
aca5b2e6a66f Merged revisions 64623,64640,64665,64687,64689-64690,64719,64721,64735,64742,64744-64746,64756-64761,64767-64769,64771-64772,64774-64775,64788,64793,64835-64836 via svnmerge from
Georg Brandl <georg@python.org>
parents: 48334
diff changeset
884 if self.port == self.default_port:
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 36559
diff changeset
885 self.putheader('Host', host_enc)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 36559
diff changeset
886 else:
43777
0d462d789b18 Merging the py3k-pep3137 branch back into the py3k branch.
Guido van Rossum <guido@python.org>
parents: 43734
diff changeset
887 host_enc = host_enc.decode("ascii")
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 36559
diff changeset
888 self.putheader('Host', "%s:%s" % (host_enc, self.port))
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
889
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
890 # note: we are assuming that clients will not attempt to set these
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
891 # headers since *this* library must deal with the
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
892 # consequences. this also means that when the supporting
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
893 # libraries are updated to recognize other forms, then this
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
894 # code should be changed (removed or updated).
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
895
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
896 # we only want a Content-Encoding of "identity" since we don't
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
897 # support encodings such as x-gzip or x-deflate.
30976
ead4c61219d0 Patch #831747: Add skip_accept_encoding parameter to putrequest.
Martin v. Löwis <martin@v.loewis.de>
parents: 30888
diff changeset
898 if not skip_accept_encoding:
ead4c61219d0 Patch #831747: Add skip_accept_encoding parameter to putrequest.
Martin v. Löwis <martin@v.loewis.de>
parents: 30888
diff changeset
899 self.putheader('Accept-Encoding', 'identity')
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
900
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
901 # we can accept "chunked" Transfer-Encodings, but no others
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
902 # NOTE: no TE header implies *only* "chunked"
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
903 #self.putheader('TE', 'chunked')
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
904
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
905 # if TE is supplied in the header, then it must appear in a
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
906 # Connection header.
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
907 #self.putheader('Connection', 'TE')
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
908
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
909 else:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
910 # For HTTP/1.0, the server will assume "not chunked"
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
911 pass
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
912
49734
b0fcb17f7efa Merged revisions 67154,67157-67159,67175-67176,67189,67224-67227,67234 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 48494
diff changeset
913 def putheader(self, header, *values):
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
914 """Send a request header line to the server.
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
915
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
916 For example: h.putheader('Accept', 'text/html')
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
917 """
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
918 if self.__state != _CS_REQ_STARTED:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
919 raise CannotSendHeader()
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
920
43777
0d462d789b18 Merging the py3k-pep3137 branch back into the py3k branch.
Guido van Rossum <guido@python.org>
parents: 43734
diff changeset
921 if hasattr(header, 'encode'):
0d462d789b18 Merging the py3k-pep3137 branch back into the py3k branch.
Guido van Rossum <guido@python.org>
parents: 43734
diff changeset
922 header = header.encode('ascii')
49734
b0fcb17f7efa Merged revisions 67154,67157-67159,67175-67176,67189,67224-67227,67234 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 48494
diff changeset
923 values = list(values)
b0fcb17f7efa Merged revisions 67154,67157-67159,67175-67176,67189,67224-67227,67234 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 48494
diff changeset
924 for i, one_value in enumerate(values):
b0fcb17f7efa Merged revisions 67154,67157-67159,67175-67176,67189,67224-67227,67234 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 48494
diff changeset
925 if hasattr(one_value, 'encode'):
b0fcb17f7efa Merged revisions 67154,67157-67159,67175-67176,67189,67224-67227,67234 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 48494
diff changeset
926 values[i] = one_value.encode('ascii')
65078
ad41fbf33e3b Fix Issue10012 - httplib headers, which are (sometimes mistakenly) int are explicitly cast to str (bytes - in py3k).
Senthil Kumaran <orsenthil@gmail.com>
parents: 65052
diff changeset
927 elif isinstance(one_value, int):
ad41fbf33e3b Fix Issue10012 - httplib headers, which are (sometimes mistakenly) int are explicitly cast to str (bytes - in py3k).
Senthil Kumaran <orsenthil@gmail.com>
parents: 65052
diff changeset
928 values[i] = str(one_value).encode('ascii')
49734
b0fcb17f7efa Merged revisions 67154,67157-67159,67175-67176,67189,67224-67227,67234 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 48494
diff changeset
929 value = b'\r\n\t'.join(values)
43777
0d462d789b18 Merging the py3k-pep3137 branch back into the py3k branch.
Guido van Rossum <guido@python.org>
parents: 43734
diff changeset
930 header = header + b': ' + value
0d462d789b18 Merging the py3k-pep3137 branch back into the py3k branch.
Guido van Rossum <guido@python.org>
parents: 43734
diff changeset
931 self._output(header)
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
932
49907
75a8caa24186 Merged revisions 67442 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 49734
diff changeset
933 def endheaders(self, message_body=None):
75a8caa24186 Merged revisions 67442 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 49734
diff changeset
934 """Indicate that the last header line has been sent to the server.
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
935
49907
75a8caa24186 Merged revisions 67442 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 49734
diff changeset
936 This method sends the request to the server. The optional
75a8caa24186 Merged revisions 67442 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 49734
diff changeset
937 message_body argument can be used to pass message body
75a8caa24186 Merged revisions 67442 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 49734
diff changeset
938 associated with the request. The message body will be sent in
75a8caa24186 Merged revisions 67442 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 49734
diff changeset
939 the same packet as the message headers if possible. The
75a8caa24186 Merged revisions 67442 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 49734
diff changeset
940 message_body should be a string.
75a8caa24186 Merged revisions 67442 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 49734
diff changeset
941 """
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
942 if self.__state == _CS_REQ_STARTED:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
943 self.__state = _CS_REQ_SENT
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
944 else:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
945 raise CannotSendHeader()
49907
75a8caa24186 Merged revisions 67442 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 49734
diff changeset
946 self._send_output(message_body)
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
947
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
948 def request(self, method, url, body=None, headers={}):
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
949 """Send a complete request to the server."""
52375
a726f7928561 Remove special logic that closes HTTPConnection socket on EPIPE.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52370
diff changeset
950 self._send_request(method, url, body, headers)
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
951
49907
75a8caa24186 Merged revisions 67442 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 49734
diff changeset
952 def _set_content_length(self, body):
75a8caa24186 Merged revisions 67442 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 49734
diff changeset
953 # Set the content-length based on the body.
75a8caa24186 Merged revisions 67442 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 49734
diff changeset
954 thelen = None
75a8caa24186 Merged revisions 67442 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 49734
diff changeset
955 try:
75a8caa24186 Merged revisions 67442 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 49734
diff changeset
956 thelen = str(len(body))
75a8caa24186 Merged revisions 67442 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 49734
diff changeset
957 except TypeError as te:
75a8caa24186 Merged revisions 67442 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 49734
diff changeset
958 # If this is a file-like object, try to
75a8caa24186 Merged revisions 67442 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 49734
diff changeset
959 # fstat its file descriptor
75a8caa24186 Merged revisions 67442 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 49734
diff changeset
960 try:
75a8caa24186 Merged revisions 67442 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 49734
diff changeset
961 thelen = str(os.fstat(body.fileno()).st_size)
75a8caa24186 Merged revisions 67442 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 49734
diff changeset
962 except (AttributeError, OSError):
75a8caa24186 Merged revisions 67442 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 49734
diff changeset
963 # Don't send a length if this failed
75a8caa24186 Merged revisions 67442 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 49734
diff changeset
964 if self.debuglevel > 0: print("Cannot stat!!")
75a8caa24186 Merged revisions 67442 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 49734
diff changeset
965
75a8caa24186 Merged revisions 67442 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 49734
diff changeset
966 if thelen is not None:
75a8caa24186 Merged revisions 67442 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 49734
diff changeset
967 self.putheader('Content-Length', thelen)
75a8caa24186 Merged revisions 67442 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 49734
diff changeset
968
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
969 def _send_request(self, method, url, body, headers):
52375
a726f7928561 Remove special logic that closes HTTPConnection socket on EPIPE.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52370
diff changeset
970 # Honor explicitly requested Host: and Accept-Encoding: headers.
33001
4fd59651fe3f SF bug 874842 and patch 997626: httplib bugs
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 32252
diff changeset
971 header_names = dict.fromkeys([k.lower() for k in headers])
4fd59651fe3f SF bug 874842 and patch 997626: httplib bugs
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 32252
diff changeset
972 skips = {}
4fd59651fe3f SF bug 874842 and patch 997626: httplib bugs
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 32252
diff changeset
973 if 'host' in header_names:
4fd59651fe3f SF bug 874842 and patch 997626: httplib bugs
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 32252
diff changeset
974 skips['skip_host'] = 1
4fd59651fe3f SF bug 874842 and patch 997626: httplib bugs
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 32252
diff changeset
975 if 'accept-encoding' in header_names:
4fd59651fe3f SF bug 874842 and patch 997626: httplib bugs
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 32252
diff changeset
976 skips['skip_accept_encoding'] = 1
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
977
33001
4fd59651fe3f SF bug 874842 and patch 997626: httplib bugs
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 32252
diff changeset
978 self.putrequest(method, url, **skips)
4fd59651fe3f SF bug 874842 and patch 997626: httplib bugs
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 32252
diff changeset
979
4fd59651fe3f SF bug 874842 and patch 997626: httplib bugs
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 32252
diff changeset
980 if body and ('content-length' not in header_names):
49907
75a8caa24186 Merged revisions 67442 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 49734
diff changeset
981 self._set_content_length(body)
40809
f340cb045bf9 - PEP 3106: dict.iterkeys(), .iteritems(), .itervalues() are now gone;
Guido van Rossum <guido@python.org>
parents: 40775
diff changeset
982 for hdr, value in headers.items():
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
983 self.putheader(hdr, value)
52366
75e3fe3c0726 Simplify complex expression.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 51953
diff changeset
984 if isinstance(body, str):
52370
8b691690babd Fix some string encoding issues with entity bodies in HTTP requests.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52369
diff changeset
985 # RFC 2616 Section 3.7.1 says that text default has a
8b691690babd Fix some string encoding issues with entity bodies in HTTP requests.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52369
diff changeset
986 # default charset of iso-8859-1.
8b691690babd Fix some string encoding issues with entity bodies in HTTP requests.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52369
diff changeset
987 body = body.encode('iso-8859-1')
52366
75e3fe3c0726 Simplify complex expression.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 51953
diff changeset
988 self.endheaders(body)
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
989
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
990 def getresponse(self):
52522
9eceb618274a Simplify the Request class. The basic components of the parsed
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52375
diff changeset
991 """Get the response from the server.
9eceb618274a Simplify the Request class. The basic components of the parsed
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52375
diff changeset
992
9eceb618274a Simplify the Request class. The basic components of the parsed
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52375
diff changeset
993 If the HTTPConnection is in the correct state, returns an
9eceb618274a Simplify the Request class. The basic components of the parsed
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52375
diff changeset
994 instance of HTTPResponse or of whatever object is returned by
9eceb618274a Simplify the Request class. The basic components of the parsed
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52375
diff changeset
995 class the response_class variable.
9eceb618274a Simplify the Request class. The basic components of the parsed
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52375
diff changeset
996
9eceb618274a Simplify the Request class. The basic components of the parsed
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52375
diff changeset
997 If a request has not been sent or if a previous response has
9eceb618274a Simplify the Request class. The basic components of the parsed
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52375
diff changeset
998 not be handled, ResponseNotReady is raised. If the HTTP
9eceb618274a Simplify the Request class. The basic components of the parsed
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52375
diff changeset
999 response indicates that the connection should be closed, then
9eceb618274a Simplify the Request class. The basic components of the parsed
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52375
diff changeset
1000 it will be closed before the response is returned. When the
9eceb618274a Simplify the Request class. The basic components of the parsed
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52375
diff changeset
1001 connection is closed, the underlying socket is closed.
9eceb618274a Simplify the Request class. The basic components of the parsed
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52375
diff changeset
1002 """
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1003
29584
da0248fcfc52 Deal with a couple XXX comments which asked questions.
Greg Stein <gstein@lyra.org>
parents: 29465
diff changeset
1004 # if a prior response has been completed, then forget about it.
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1005 if self.__response and self.__response.isclosed():
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1006 self.__response = None
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1007
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1008 # if a prior response exists, then it must be completed (otherwise, we
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1009 # cannot read this response's header to determine the connection-close
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1010 # behavior)
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1011 #
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1012 # note: if a prior response existed, but was connection-close, then the
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1013 # socket and response were made independent of this HTTPConnection
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1014 # object since a new request requires that we open a whole new
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1015 # connection
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1016 #
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1017 # this means the prior response had one of two states:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1018 # 1) will_close: this connection was reset and the prior socket and
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1019 # response operate independently
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1020 # 2) persistent: the response was retained and we await its
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1021 # isclosed() status to become true.
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1022 #
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1023 if self.__state != _CS_REQ_SENT or self.__response:
52522
9eceb618274a Simplify the Request class. The basic components of the parsed
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52375
diff changeset
1024 raise ResponseNotReady(self.__state)
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1025
14672
1b20ad6eb501 Do not close socket when a Content-Length is 0. This make the
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14560
diff changeset
1026 if self.debuglevel > 0:
24119
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1027 response = self.response_class(self.sock, self.debuglevel,
29087
526f37fb380d Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 28978
diff changeset
1028 strict=self.strict,
28978
83fe0f5c9359 SF bug 622042: Don't expect response body from HEAD request.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 28671
diff changeset
1029 method=self._method)
14672
1b20ad6eb501 Do not close socket when a Content-Length is 0. This make the
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 14560
diff changeset
1030 else:
28978
83fe0f5c9359 SF bug 622042: Don't expect response body from HEAD request.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 28671
diff changeset
1031 response = self.response_class(self.sock, strict=self.strict,
83fe0f5c9359 SF bug 622042: Don't expect response body from HEAD request.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 28671
diff changeset
1032 method=self._method)
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1033
24216
c8a19a428358 Change _begin() back to begin().
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24168
diff changeset
1034 response.begin()
24038
9377158652dd Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23620
diff changeset
1035 assert response.will_close != _UNKNOWN
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1036 self.__state = _CS_IDLE
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1037
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1038 if response.will_close:
41554
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 41501
diff changeset
1039 # this effectively passes the connection to the response
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 41501
diff changeset
1040 self.close()
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1041 else:
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1042 # remember this, so we can tell when it is complete
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1043 self.__response = response
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1044
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1045 return response
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1046
43095
4a07a288f5c4 Merged revisions 57620-57771 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 42964
diff changeset
1047 try:
4a07a288f5c4 Merged revisions 57620-57771 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 42964
diff changeset
1048 import ssl
4a07a288f5c4 Merged revisions 57620-57771 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 42964
diff changeset
1049 except ImportError:
4a07a288f5c4 Merged revisions 57620-57771 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 42964
diff changeset
1050 pass
4a07a288f5c4 Merged revisions 57620-57771 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 42964
diff changeset
1051 else:
4a07a288f5c4 Merged revisions 57620-57771 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 42964
diff changeset
1052 class HTTPSConnection(HTTPConnection):
4a07a288f5c4 Merged revisions 57620-57771 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 42964
diff changeset
1053 "This class allows communication via SSL."
14140
f7a8c40b3880 add docstring explaining makefile limitation
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 13787
diff changeset
1054
43095
4a07a288f5c4 Merged revisions 57620-57771 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 42964
diff changeset
1055 default_port = HTTPS_PORT
24038
9377158652dd Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23620
diff changeset
1056
65247
12327b9d3325 Issue #9003: http.client.HTTPSConnection, urllib.request.HTTPSHandler and
Antoine Pitrou <solipsis@pitrou.net>
parents: 65149
diff changeset
1057 # XXX Should key_file and cert_file be deprecated in favour of context?
12327b9d3325 Issue #9003: http.client.HTTPSConnection, urllib.request.HTTPSHandler and
Antoine Pitrou <solipsis@pitrou.net>
parents: 65149
diff changeset
1058
43095
4a07a288f5c4 Merged revisions 57620-57771 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 42964
diff changeset
1059 def __init__(self, host, port=None, key_file=None, cert_file=None,
58114
b0386fe650c7 Merged revisions 77263-77264 via svnmerge from
Gregory P. Smith <greg@mad-scientist.com>
parents: 57801
diff changeset
1060 strict=None, timeout=socket._GLOBAL_DEFAULT_TIMEOUT,
65247
12327b9d3325 Issue #9003: http.client.HTTPSConnection, urllib.request.HTTPSHandler and
Antoine Pitrou <solipsis@pitrou.net>
parents: 65149
diff changeset
1061 source_address=None, *, context=None, check_hostname=None):
58114
b0386fe650c7 Merged revisions 77263-77264 via svnmerge from
Gregory P. Smith <greg@mad-scientist.com>
parents: 57801
diff changeset
1062 super(HTTPSConnection, self).__init__(host, port, strict, timeout,
b0386fe650c7 Merged revisions 77263-77264 via svnmerge from
Gregory P. Smith <greg@mad-scientist.com>
parents: 57801
diff changeset
1063 source_address)
43095
4a07a288f5c4 Merged revisions 57620-57771 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 42964
diff changeset
1064 self.key_file = key_file
4a07a288f5c4 Merged revisions 57620-57771 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 42964
diff changeset
1065 self.cert_file = cert_file
65247
12327b9d3325 Issue #9003: http.client.HTTPSConnection, urllib.request.HTTPSHandler and
Antoine Pitrou <solipsis@pitrou.net>
parents: 65149
diff changeset
1066 if context is None:
12327b9d3325 Issue #9003: http.client.HTTPSConnection, urllib.request.HTTPSHandler and
Antoine Pitrou <solipsis@pitrou.net>
parents: 65149
diff changeset
1067 # Some reasonable defaults
12327b9d3325 Issue #9003: http.client.HTTPSConnection, urllib.request.HTTPSHandler and
Antoine Pitrou <solipsis@pitrou.net>
parents: 65149
diff changeset
1068 context = ssl.SSLContext(ssl.PROTOCOL_SSLv23)
12327b9d3325 Issue #9003: http.client.HTTPSConnection, urllib.request.HTTPSHandler and
Antoine Pitrou <solipsis@pitrou.net>
parents: 65149
diff changeset
1069 context.options |= ssl.OP_NO_SSLv2
12327b9d3325 Issue #9003: http.client.HTTPSConnection, urllib.request.HTTPSHandler and
Antoine Pitrou <solipsis@pitrou.net>
parents: 65149
diff changeset
1070 will_verify = context.verify_mode != ssl.CERT_NONE
12327b9d3325 Issue #9003: http.client.HTTPSConnection, urllib.request.HTTPSHandler and
Antoine Pitrou <solipsis@pitrou.net>
parents: 65149
diff changeset
1071 if check_hostname is None:
12327b9d3325 Issue #9003: http.client.HTTPSConnection, urllib.request.HTTPSHandler and
Antoine Pitrou <solipsis@pitrou.net>
parents: 65149
diff changeset
1072 check_hostname = will_verify
12327b9d3325 Issue #9003: http.client.HTTPSConnection, urllib.request.HTTPSHandler and
Antoine Pitrou <solipsis@pitrou.net>
parents: 65149
diff changeset
1073 elif check_hostname and not will_verify:
12327b9d3325 Issue #9003: http.client.HTTPSConnection, urllib.request.HTTPSHandler and
Antoine Pitrou <solipsis@pitrou.net>
parents: 65149
diff changeset
1074 raise ValueError("check_hostname needs a SSL context with "
12327b9d3325 Issue #9003: http.client.HTTPSConnection, urllib.request.HTTPSHandler and
Antoine Pitrou <solipsis@pitrou.net>
parents: 65149
diff changeset
1075 "either CERT_OPTIONAL or CERT_REQUIRED")
12327b9d3325 Issue #9003: http.client.HTTPSConnection, urllib.request.HTTPSHandler and
Antoine Pitrou <solipsis@pitrou.net>
parents: 65149
diff changeset
1076 if key_file or cert_file:
65249
a1567893783f Typo. It is rather obvious that this code isn't tested :/
Antoine Pitrou <solipsis@pitrou.net>
parents: 65247
diff changeset
1077 context.load_cert_chain(cert_file, key_file)
65247
12327b9d3325 Issue #9003: http.client.HTTPSConnection, urllib.request.HTTPSHandler and
Antoine Pitrou <solipsis@pitrou.net>
parents: 65149
diff changeset
1078 self._context = context
12327b9d3325 Issue #9003: http.client.HTTPSConnection, urllib.request.HTTPSHandler and
Antoine Pitrou <solipsis@pitrou.net>
parents: 65149
diff changeset
1079 self._check_hostname = check_hostname
24038
9377158652dd Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23620
diff changeset
1080
43095
4a07a288f5c4 Merged revisions 57620-57771 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 42964
diff changeset
1081 def connect(self):
4a07a288f5c4 Merged revisions 57620-57771 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 42964
diff changeset
1082 "Connect to a host on a given (SSL) port."
24168
fdd836575248 Fix for SF bug 579107.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24123
diff changeset
1083
52375
a726f7928561 Remove special logic that closes HTTPConnection socket on EPIPE.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 52370
diff changeset
1084 sock = socket.create_connection((self.host, self.port),
58114
b0386fe650c7 Merged revisions 77263-77264 via svnmerge from
Gregory P. Smith <greg@mad-scientist.com>
parents: 57801
diff changeset
1085 self.timeout, self.source_address)
55438
d1ec95ae0ac2 Fixed Issue1424152 in Py3k: urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 52631
diff changeset
1086
d1ec95ae0ac2 Fixed Issue1424152 in Py3k: urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 52631
diff changeset
1087 if self._tunnel_host:
d1ec95ae0ac2 Fixed Issue1424152 in Py3k: urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 52631
diff changeset
1088 self.sock = sock
d1ec95ae0ac2 Fixed Issue1424152 in Py3k: urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 52631
diff changeset
1089 self._tunnel()
d1ec95ae0ac2 Fixed Issue1424152 in Py3k: urllib2 fails with HTTPS over Proxy.
Senthil Kumaran <orsenthil@gmail.com>
parents: 52631
diff changeset
1090
65593
846c0e1342d0 Issue #5639: Add a *server_hostname* argument to `SSLContext.wrap_socket`
Antoine Pitrou <solipsis@pitrou.net>
parents: 65249
diff changeset
1091 server_hostname = self.host if ssl.HAS_SNI else None
846c0e1342d0 Issue #5639: Add a *server_hostname* argument to `SSLContext.wrap_socket`
Antoine Pitrou <solipsis@pitrou.net>
parents: 65249
diff changeset
1092 self.sock = self._context.wrap_socket(sock,
846c0e1342d0 Issue #5639: Add a *server_hostname* argument to `SSLContext.wrap_socket`
Antoine Pitrou <solipsis@pitrou.net>
parents: 65249
diff changeset
1093 server_hostname=server_hostname)
65247
12327b9d3325 Issue #9003: http.client.HTTPSConnection, urllib.request.HTTPSHandler and
Antoine Pitrou <solipsis@pitrou.net>
parents: 65149
diff changeset
1094 try:
12327b9d3325 Issue #9003: http.client.HTTPSConnection, urllib.request.HTTPSHandler and
Antoine Pitrou <solipsis@pitrou.net>
parents: 65149
diff changeset
1095 if self._check_hostname:
12327b9d3325 Issue #9003: http.client.HTTPSConnection, urllib.request.HTTPSHandler and
Antoine Pitrou <solipsis@pitrou.net>
parents: 65149
diff changeset
1096 ssl.match_hostname(self.sock.getpeercert(), self.host)
12327b9d3325 Issue #9003: http.client.HTTPSConnection, urllib.request.HTTPSHandler and
Antoine Pitrou <solipsis@pitrou.net>
parents: 65149
diff changeset
1097 except Exception:
12327b9d3325 Issue #9003: http.client.HTTPSConnection, urllib.request.HTTPSHandler and
Antoine Pitrou <solipsis@pitrou.net>
parents: 65149
diff changeset
1098 self.sock.shutdown(socket.SHUT_RDWR)
12327b9d3325 Issue #9003: http.client.HTTPSConnection, urllib.request.HTTPSHandler and
Antoine Pitrou <solipsis@pitrou.net>
parents: 65149
diff changeset
1099 self.sock.close()
12327b9d3325 Issue #9003: http.client.HTTPSConnection, urllib.request.HTTPSHandler and
Antoine Pitrou <solipsis@pitrou.net>
parents: 65149
diff changeset
1100 raise
24038
9377158652dd Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 23620
diff changeset
1101
43095
4a07a288f5c4 Merged revisions 57620-57771 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 42964
diff changeset
1102 __all__.append("HTTPSConnection")
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1103
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1104 class HTTPException(Exception):
24120
b51a578f5769 Fix SF bug #575360
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24119
diff changeset
1105 # Subclasses that define an __init__ must call Exception.__init__
b51a578f5769 Fix SF bug #575360
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24119
diff changeset
1106 # or define self.args. Otherwise, str() will fail.
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1107 pass
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1108
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1109 class NotConnected(HTTPException):
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1110 pass
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1111
22494
7224a1df4809 add InvalidURL exception - raised if port is given but empty or non-numeric
Skip Montanaro <skip@pobox.com>
parents: 22403
diff changeset
1112 class InvalidURL(HTTPException):
7224a1df4809 add InvalidURL exception - raised if port is given but empty or non-numeric
Skip Montanaro <skip@pobox.com>
parents: 22403
diff changeset
1113 pass
7224a1df4809 add InvalidURL exception - raised if port is given but empty or non-numeric
Skip Montanaro <skip@pobox.com>
parents: 22403
diff changeset
1114
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1115 class UnknownProtocol(HTTPException):
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1116 def __init__(self, version):
24120
b51a578f5769 Fix SF bug #575360
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24119
diff changeset
1117 self.args = version,
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1118 self.version = version
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1119
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1120 class UnknownTransferEncoding(HTTPException):
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1121 pass
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1122
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1123 class UnimplementedFileMode(HTTPException):
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1124 pass
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1125
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1126 class IncompleteRead(HTTPException):
51953
9c56eb17f334 Merged revisions 70107 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 51532
diff changeset
1127 def __init__(self, partial, expected=None):
24120
b51a578f5769 Fix SF bug #575360
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24119
diff changeset
1128 self.args = partial,
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1129 self.partial = partial
51953
9c56eb17f334 Merged revisions 70107 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 51532
diff changeset
1130 self.expected = expected
9c56eb17f334 Merged revisions 70107 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 51532
diff changeset
1131 def __repr__(self):
9c56eb17f334 Merged revisions 70107 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 51532
diff changeset
1132 if self.expected is not None:
9c56eb17f334 Merged revisions 70107 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 51532
diff changeset
1133 e = ', %i more expected' % self.expected
9c56eb17f334 Merged revisions 70107 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 51532
diff changeset
1134 else:
9c56eb17f334 Merged revisions 70107 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 51532
diff changeset
1135 e = ''
9c56eb17f334 Merged revisions 70107 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 51532
diff changeset
1136 return 'IncompleteRead(%i bytes read%s)' % (len(self.partial), e)
9c56eb17f334 Merged revisions 70107 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 51532
diff changeset
1137 def __str__(self):
9c56eb17f334 Merged revisions 70107 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 51532
diff changeset
1138 return repr(self)
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1139
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1140 class ImproperConnectionState(HTTPException):
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1141 pass
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1142
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1143 class CannotSendRequest(ImproperConnectionState):
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1144 pass
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1145
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1146 class CannotSendHeader(ImproperConnectionState):
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1147 pass
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1148
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1149 class ResponseNotReady(ImproperConnectionState):
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1150 pass
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1151
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1152 class BadStatusLine(HTTPException):
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1153 def __init__(self, line):
59875
d9b8f273ab8d Merged revisions 78417 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 58114
diff changeset
1154 if not line:
d9b8f273ab8d Merged revisions 78417 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 58114
diff changeset
1155 line = repr(line)
24120
b51a578f5769 Fix SF bug #575360
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24119
diff changeset
1156 self.args = line,
13636
b5120f58c5da no changes other than indentation level (now 4) and comment reflow.
Greg Stein <gstein@lyra.org>
parents: 13019
diff changeset
1157 self.line = line
13019
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1158
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1159 # for backwards compatibility
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1160 error = HTTPException
4a3e9871b41b initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
Greg Stein <gstein@lyra.org>
parents: 12716
diff changeset
1161
24119
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1162 class LineAndFileWrapper:
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1163 """A limited file-like object for HTTP/0.9 responses."""
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1164
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1165 # The status-line parsing code calls readline(), which normally
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1166 # get the HTTP status line. For a 0.9 response, however, this is
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1167 # actually the first line of the body! Clients need to get a
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1168 # readable file object that contains that line.
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1169
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1170 def __init__(self, line, file):
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1171 self._line = line
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1172 self._file = file
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1173 self._line_consumed = 0
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1174 self._line_offset = 0
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1175 self._line_left = len(line)
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1176
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1177 def __getattr__(self, attr):
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1178 return getattr(self._file, attr)
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1179
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1180 def _done(self):
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1181 # called when the last byte is read from the line. After the
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1182 # call, all read methods are delegated to the underlying file
28063
Skip Montanaro <skip@pobox.com>
parents: 26387
diff changeset
1183 # object.
24119
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1184 self._line_consumed = 1
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1185 self.read = self._file.read
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1186 self.readline = self._file.readline
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1187 self.readlines = self._file.readlines
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1188
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1189 def read(self, amt=None):
32252
a72bc4257336 Fix a bug that robotparser starves memory when the server responses
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31967
diff changeset
1190 if self._line_consumed:
a72bc4257336 Fix a bug that robotparser starves memory when the server responses
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31967
diff changeset
1191 return self._file.read(amt)
a72bc4257336 Fix a bug that robotparser starves memory when the server responses
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31967
diff changeset
1192 assert self._line_left
24119
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1193 if amt is None or amt > self._line_left:
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1194 s = self._line[self._line_offset:]
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1195 self._done()
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1196 if amt is None:
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1197 return s + self._file.read()
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1198 else:
24263
e6033254df21 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 24262
diff changeset
1199 return s + self._file.read(amt - len(s))
24119
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1200 else:
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1201 assert amt <= self._line_left
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1202 i = self._line_offset
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1203 j = i + amt
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1204 s = self._line[i:j]
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1205 self._line_offset = j
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1206 self._line_left -= amt
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1207 if self._line_left == 0:
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1208 self._done()
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1209 return s
24263
e6033254df21 Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 24262
diff changeset
1210
24119
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1211 def readline(self):
32252
a72bc4257336 Fix a bug that robotparser starves memory when the server responses
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31967
diff changeset
1212 if self._line_consumed:
a72bc4257336 Fix a bug that robotparser starves memory when the server responses
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31967
diff changeset
1213 return self._file.readline()
a72bc4257336 Fix a bug that robotparser starves memory when the server responses
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31967
diff changeset
1214 assert self._line_left
24119
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1215 s = self._line[self._line_offset:]
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1216 self._done()
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1217 return s
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1218
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1219 def readlines(self, size=None):
32252
a72bc4257336 Fix a bug that robotparser starves memory when the server responses
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31967
diff changeset
1220 if self._line_consumed:
a72bc4257336 Fix a bug that robotparser starves memory when the server responses
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31967
diff changeset
1221 return self._file.readlines(size)
a72bc4257336 Fix a bug that robotparser starves memory when the server responses
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31967
diff changeset
1222 assert self._line_left
24119
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1223 L = [self._line[self._line_offset:]]
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1224 self._done()
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1225 if size is None:
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1226 return L + self._file.readlines()
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1227 else:
f33bb9c131f8 Handle HTTP/0.9 responses.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24088
diff changeset
1228 return L + self._file.readlines(size)