Mercurial > cpython-lru_cache-weakref
annotate Lib/SocketServer.py @ 5817:48b94de85bf3 legacy-trunk
Extensive changes by AMK.
| author | Guido van Rossum <guido@python.org> |
|---|---|
| date | Thu, 17 Jul 1997 16:29:42 +0000 |
| parents | b56d527a433f |
| children | a5f29a560c09 |
| rev | line source |
|---|---|
| 2887 | 1 """Generic socket server classes. |
| 2 | |
| 3 This module tries to capture the various aspects of defining a server: | |
| 4 | |
| 5 - address family: | |
| 6 - AF_INET: IP (Internet Protocol) sockets (default) | |
| 7 - AF_UNIX: Unix domain sockets | |
| 8 - others, e.g. AF_DECNET are conceivable (see <socket.h> | |
| 9 - socket type: | |
| 10 - SOCK_STREAM (reliable stream, e.g. TCP) | |
| 11 - SOCK_DGRAM (datagrams, e.g. UDP) | |
| 12 - client address verification before further looking at the request | |
| 13 (This is actually a hook for any processing that needs to look | |
| 14 at the request before anything else, e.g. logging) | |
| 15 - how to handle multiple requests: | |
| 16 - synchronous (one request is handled at a time) | |
| 17 - forking (each request is handled by a new process) | |
| 18 - threading (each request is handled by a new thread) | |
| 19 | |
| 20 The classes in this module favor the server type that is simplest to | |
| 21 write: a synchronous TCP/IP server. This is bad class design, but | |
| 22 save some typing. (There's also the issue that a deep class hierarchy | |
| 23 slows down method lookups.) | |
| 24 | |
| 25 There are four classes in an inheritance diagram that represent | |
| 26 synchronous servers of four types: | |
| 27 | |
| 28 +-----------+ +------------------+ | |
| 29 | TCPServer |------->| UnixStreamServer | | |
| 30 +-----------+ +------------------+ | |
| 31 | | |
| 32 v | |
| 33 +-----------+ +--------------------+ | |
| 34 | UDPServer |------->| UnixDatagramServer | | |
| 35 +-----------+ +--------------------+ | |
| 36 | |
|
5803
b56d527a433f
Fix bug found by kjpylint; change doc string to avoid "(" in column 1.
Guido van Rossum <guido@python.org>
parents:
4459
diff
changeset
|
37 Note that UnixDatagramServer derives from UDPServer, not from |
| 2887 | 38 UnixStreamServer -- the only difference between an IP and a Unix |
| 39 stream server is the address family, which is simply repeated in both | |
|
5803
b56d527a433f
Fix bug found by kjpylint; change doc string to avoid "(" in column 1.
Guido van Rossum <guido@python.org>
parents:
4459
diff
changeset
|
40 unix server classes. |
| 2887 | 41 |
| 42 Forking and threading versions of each type of server can be created | |
| 43 using the ForkingServer and ThreadingServer mix-in classes. For | |
| 44 instance, a threading UDP server class is created as follows: | |
| 45 | |
| 46 class ThreadingUDPServer(ThreadingMixIn, UDPServer): pass | |
| 47 | |
|
5803
b56d527a433f
Fix bug found by kjpylint; change doc string to avoid "(" in column 1.
Guido van Rossum <guido@python.org>
parents:
4459
diff
changeset
|
48 The Mix-in class must come first, since it overrides a method defined |
|
b56d527a433f
Fix bug found by kjpylint; change doc string to avoid "(" in column 1.
Guido van Rossum <guido@python.org>
parents:
4459
diff
changeset
|
49 in UDPServer! |
| 2887 | 50 |
| 51 To implement a service, you must derive a class from | |
| 52 BaseRequestHandler and redefine its handle() method. You can then run | |
| 53 various versions of the service by combining one of the server classes | |
| 54 with your request handler class. | |
| 55 | |
| 56 The request handler class must be different for datagram or stream | |
| 57 services. This can be hidden by using the mix-in request handler | |
| 58 classes StreamRequestHandler or DatagramRequestHandler. | |
| 59 | |
| 60 Of course, you still have to use your head! | |
| 61 | |
| 62 For instance, it makes no sense to use a forking server if the service | |
| 63 contains state in memory that can be modified by requests (since the | |
| 64 modifications in the child process would never reach the initial state | |
| 65 kept in the parent process and passed to each child). In this case, | |
| 66 you can use a threading server, but you will probably have to use | |
| 67 locks to avoid two requests that come in nearly simultaneous to apply | |
| 68 conflicting changes to the server state. | |
| 69 | |
| 70 On the other hand, if you are building e.g. an HTTP server, where all | |
| 71 data is stored externally (e.g. in the file system), a synchronous | |
| 72 class will essentially render the service "deaf" while one request is | |
| 73 being handled -- which may be for a very long time if a client is slow | |
| 74 to reqd all the data it has requested. Here a threading or forking | |
| 75 server is appropriate. | |
| 76 | |
| 77 In some cases, it may be appropriate to process part of a request | |
| 78 synchronously, but to finish processing in a forked child depending on | |
| 79 the request data. This can be implemented by using a synchronous | |
| 80 server and doing an explicit fork in the request handler class's | |
| 81 handle() method. | |
| 82 | |
| 83 Another approach to handling multiple simultaneous requests in an | |
| 84 environment that supports neither threads nor fork (or where these are | |
| 85 too expensive or inappropriate for the service) is to maintain an | |
| 86 explicit table of partially finished requests and to use select() to | |
| 87 decide which request to work on next (or whether to handle a new | |
| 88 incoming request). This is particularly important for stream services | |
| 89 where each client can potentially be connected for a long time (if | |
| 90 threads or subprocesses can't be used). | |
| 91 | |
| 92 Future work: | |
| 93 - Standard classes for Sun RPC (which uses either UDP or TCP) | |
| 94 - Standard mix-in classes to implement various authentication | |
| 95 and encryption schemes | |
| 96 - Standard framework for select-based multiplexing | |
| 97 | |
| 98 XXX Open problems: | |
| 99 - What to do with out-of-band data? | |
| 100 | |
| 101 """ | |
| 102 | |
| 103 | |
| 104 __version__ = "0.2" | |
| 105 | |
| 106 | |
| 107 import socket | |
| 108 import sys | |
| 109 import os | |
| 110 | |
| 111 | |
| 112 class TCPServer: | |
| 113 | |
| 114 """Base class for various socket-based server classes. | |
| 115 | |
| 116 Defaults to synchronous IP stream (i.e., TCP). | |
| 117 | |
| 118 Methods for the caller: | |
| 119 | |
| 120 - __init__(server_address, RequestHandlerClass) | |
| 121 - serve_forever() | |
| 122 - handle_request() # if you don't use serve_forever() | |
| 123 - fileno() -> int # for select() | |
| 124 | |
| 125 Methods that may be overridden: | |
| 126 | |
| 127 - server_bind() | |
| 128 - server_activate() | |
| 129 - get_request() -> request, client_address | |
| 130 - verify_request(request, client_address) | |
| 131 - process_request(request, client_address) | |
| 132 - handle_error() | |
| 133 | |
| 134 Methods for derived classes: | |
| 135 | |
| 136 - finish_request(request, client_address) | |
| 137 | |
| 138 Class variables that may be overridden by derived classes or | |
| 139 instances: | |
| 140 | |
| 141 - address_family | |
| 142 - socket_type | |
| 143 - request_queue_size (only for stream sockets) | |
| 144 | |
| 145 Instance variables: | |
| 146 | |
| 147 - server_address | |
| 148 - RequestHandlerClass | |
| 149 - socket | |
| 150 | |
| 151 """ | |
| 152 | |
| 153 address_family = socket.AF_INET | |
| 154 | |
| 155 socket_type = socket.SOCK_STREAM | |
| 156 | |
| 157 request_queue_size = 5 | |
| 158 | |
| 159 def __init__(self, server_address, RequestHandlerClass): | |
| 160 """Constructor. May be extended, do not override.""" | |
| 161 self.server_address = server_address | |
| 162 self.RequestHandlerClass = RequestHandlerClass | |
| 163 self.socket = socket.socket(self.address_family, | |
| 164 self.socket_type) | |
| 165 self.server_bind() | |
| 166 self.server_activate() | |
| 167 | |
| 168 def server_bind(self): | |
| 169 """Called by constructor to bind the socket. | |
| 170 | |
| 171 May be overridden. | |
| 172 | |
| 173 """ | |
| 174 self.socket.bind(self.server_address) | |
| 175 | |
| 176 def server_activate(self): | |
| 177 """Called by constructor to activate the server. | |
| 178 | |
| 179 May be overridden. | |
| 180 | |
| 181 """ | |
| 182 self.socket.listen(self.request_queue_size) | |
| 183 | |
| 184 def fileno(self): | |
| 185 """Return socket file number. | |
| 186 | |
| 187 Interface required by select(). | |
| 188 | |
| 189 """ | |
| 190 return self.socket.fileno() | |
| 191 | |
| 192 def serve_forever(self): | |
| 193 """Handle one request at a time until doomsday.""" | |
| 194 while 1: | |
| 195 self.handle_request() | |
| 196 | |
| 197 # The distinction between handling, getting, processing and | |
| 198 # finishing a request is fairly arbitrary. Remember: | |
| 199 # | |
| 200 # - handle_request() is the top-level call. It calls | |
| 201 # get_request(), verify_request() and process_request() | |
| 202 # - get_request() is different for stream or datagram sockets | |
| 203 # - process_request() is the place that may fork a new process | |
| 204 # or create a new thread to finish the request | |
| 205 # - finish_request() instantiates the request handler class; | |
| 206 # this constructor will handle the request all by itself | |
| 207 | |
| 208 def handle_request(self): | |
| 209 """Handle one request, possibly blocking.""" | |
| 210 request, client_address = self.get_request() | |
| 211 if self.verify_request(request, client_address): | |
| 212 try: | |
| 213 self.process_request(request, client_address) | |
| 214 except: | |
| 215 self.handle_error(request, client_address) | |
| 216 | |
| 217 def get_request(self): | |
| 218 """Get the request and client address from the socket. | |
| 219 | |
| 220 May be overridden. | |
| 221 | |
| 222 """ | |
| 223 return self.socket.accept() | |
| 224 | |
| 225 def verify_request(self, request, client_address): | |
| 226 """Verify the request. May be overridden. | |
| 227 | |
| 228 Return true if we should proceed with this request. | |
| 229 | |
| 230 """ | |
| 231 return 1 | |
| 232 | |
| 233 def process_request(self, request, client_address): | |
| 234 """Call finish_request. | |
| 235 | |
| 236 Overridden by ForkingMixIn and ThreadingMixIn. | |
| 237 | |
| 238 """ | |
| 239 self.finish_request(request, client_address) | |
| 240 | |
| 241 def finish_request(self, request, client_address): | |
| 242 """Finish one request by instantiating RequestHandlerClass.""" | |
| 243 self.RequestHandlerClass(request, client_address, self) | |
| 244 | |
| 245 def handle_error(self, request, client_address): | |
| 246 """Handle an error gracefully. May be overridden. | |
| 247 | |
| 248 The default is to print a traceback and continue. | |
| 249 | |
| 250 """ | |
| 251 exc, value, tb = sys.exc_type, sys.exc_value, sys.exc_traceback | |
| 252 print '-'*40 | |
| 253 print 'Exception happened during processing of request from', | |
| 254 print client_address | |
| 255 import traceback | |
| 256 traceback.print_exception(exc, value, tb) | |
| 257 print '-'*40 | |
| 258 | |
| 259 | |
| 260 class UDPServer(TCPServer): | |
| 261 | |
| 262 """UDP server class.""" | |
| 263 | |
| 264 socket_type = socket.SOCK_DGRAM | |
| 265 | |
| 266 max_packet_size = 8192 | |
| 267 | |
| 268 def get_request(self): | |
|
5803
b56d527a433f
Fix bug found by kjpylint; change doc string to avoid "(" in column 1.
Guido van Rossum <guido@python.org>
parents:
4459
diff
changeset
|
269 return self.socket.recvfrom(self.max_packet_size) |
| 2887 | 270 |
| 271 | |
| 272 if hasattr(socket, 'AF_UNIX'): | |
| 273 | |
| 274 class UnixStreamServer(TCPServer): | |
| 275 | |
| 276 address_family = socket.AF_UNIX | |
| 277 | |
| 278 | |
| 279 class UnixDatagramServer(UDPServer): | |
| 280 | |
| 281 address_family = socket.AF_UNIX | |
| 282 | |
| 283 | |
| 284 class ForkingMixIn: | |
| 285 | |
| 286 """Mix-in class to handle each request in a new process.""" | |
| 287 | |
| 288 active_children = None | |
| 289 | |
| 290 def collect_children(self): | |
| 291 """Internal routine to wait for died children.""" | |
| 292 while self.active_children: | |
|
3354
0b021cf41dda
correct retrieving return value of os.waitpid()
Guido van Rossum <guido@python.org>
parents:
2887
diff
changeset
|
293 pid, status = os.waitpid(0, os.WNOHANG) |
| 2887 | 294 if not pid: break |
| 295 self.active_children.remove(pid) | |
| 296 | |
| 297 def process_request(self, request, client_address): | |
| 298 """Fork a new subprocess to process the request.""" | |
| 299 self.collect_children() | |
| 300 pid = os.fork() | |
| 301 if pid: | |
| 302 # Parent process | |
| 303 if self.active_children is None: | |
| 304 self.active_children = [] | |
| 305 self.active_children.append(pid) | |
| 306 return | |
| 307 else: | |
| 308 # Child process. | |
| 309 # This must never return, hence os._exit()! | |
| 310 try: | |
| 311 self.finish_request(request, client_address) | |
| 312 os._exit(0) | |
| 313 except: | |
| 314 try: | |
| 315 self.handle_error(request, | |
| 316 client_address) | |
| 317 finally: | |
| 318 os._exit(1) | |
| 319 | |
| 320 | |
| 321 class ThreadingMixIn: | |
| 322 | |
| 323 """Mix-in class to handle each request in a new thread.""" | |
| 324 | |
| 325 def process_request(self, request, client_address): | |
| 326 """Start a new thread to process the request.""" | |
| 327 import thread | |
| 328 thread.start_new_thread(self.finish_request, | |
| 329 (request, client_address)) | |
| 330 | |
| 331 | |
| 332 class ForkingUDPServer(ForkingMixIn, UDPServer): pass | |
| 333 class ForkingTCPServer(ForkingMixIn, TCPServer): pass | |
| 334 | |
| 335 class ThreadingUDPServer(ThreadingMixIn, UDPServer): pass | |
| 336 class ThreadingTCPServer(ThreadingMixIn, TCPServer): pass | |
| 337 | |
| 338 | |
| 339 class BaseRequestHandler: | |
| 340 | |
| 341 """Base class for request handler classes. | |
| 342 | |
| 343 This class is instantiated for each request to be handled. The | |
| 344 constructor sets the instance variables request, client_address | |
| 345 and server, and then calls the handle() method. To implement a | |
| 346 specific service, all you need to do is to derive a class which | |
| 347 defines a handle() method. | |
| 348 | |
| 349 The handle() method can find the request as self.request, the | |
| 350 client address as self.client_request, and the server (in case it | |
| 351 needs access to per-server information) as self.server. Since a | |
| 352 separate instance is created for each request, the handle() method | |
| 353 can define arbitrary other instance variariables. | |
| 354 | |
| 355 """ | |
| 356 | |
| 357 def __init__(self, request, client_address, server): | |
| 358 self.request = request | |
| 359 self.client_address = client_address | |
| 360 self.server = server | |
| 361 try: | |
| 362 self.setup() | |
| 363 self.handle() | |
| 364 self.finish() | |
| 365 finally: | |
| 366 sys.exc_traceback = None # Help garbage collection | |
| 367 | |
| 368 def setup(self): | |
| 369 pass | |
| 370 | |
| 371 def __del__(self): | |
| 372 pass | |
| 373 | |
| 374 def handle(self): | |
| 375 pass | |
| 376 | |
| 377 def finish(self): | |
| 378 pass | |
| 379 | |
| 380 | |
| 381 # The following two classes make it possible to use the same service | |
| 382 # class for stream or datagram servers. | |
| 383 # Each class sets up these instance variables: | |
| 384 # - rfile: a file object from which receives the request is read | |
| 385 # - wfile: a file object to which the reply is written | |
| 386 # When the handle() method returns, wfile is flushed properly | |
| 387 | |
| 388 | |
| 389 class StreamRequestHandler(BaseRequestHandler): | |
| 390 | |
| 391 """Define self.rfile and self.wfile for stream sockets.""" | |
| 392 | |
| 393 def setup(self): | |
| 394 self.connection = self.request | |
|
4459
c91aacbf8ed4
Make self.rfile unbuffered (self.wfile already is). This should fix
Guido van Rossum <guido@python.org>
parents:
3398
diff
changeset
|
395 self.rfile = self.connection.makefile('rb', 0) |
|
3398
409a89c5f66d
Changed makefile() args from r/w to rb/wb, for non-unix compatability.
Jack Jansen <jack.jansen@cwi.nl>
parents:
3354
diff
changeset
|
396 self.wfile = self.connection.makefile('wb', 0) |
| 2887 | 397 |
| 398 def finish(self): | |
| 399 self.wfile.flush() | |
| 400 | |
| 401 | |
| 402 class DatagramRequestHandler(BaseRequestHandler): | |
| 403 | |
| 404 """Define self.rfile and self.wfile for datagram sockets.""" | |
| 405 | |
| 406 def setup(self): | |
| 407 import StringIO | |
| 408 self.packet, self.socket = self.request | |
| 409 self.rfile = StringIO.StringIO(self.packet) | |
| 410 self.wfile = StringIO.StringIO(self.packet) | |
| 411 | |
| 412 def finish(self): | |
| 413 self.socket.send(self.wfile.getvalue()) |