Mercurial > cpython-lru_cache-weakref

annotate Lib/multiprocessing/managers.py @ 87690:66c1c9f32567 default tip

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Add use_weakref option to functools.lru_cache The use_weakref option makes lru_cache create weak references to the function arguments when creating the cache key, which prevents them from staying in the memory forever even after all other references to them are gone. This is a proof of concept for http://bugs.python.org/issue19859
author Radomir Dopieralski <devel@sheep.art.pl>
date Tue, 03 Dec 2013 16:02:48 +0100
parents 72a5ac909c7a
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1 #
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
2 # Module providing the `SyncManager` class for dealing
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
3 # with shared objects
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
4 #
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
5 # multiprocessing/managers.py
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
6 #
66886
71fb03274a89 Merged revisions 87225 via svnmerge from
R. David Murray <rdmurray@bitdance.com>
parents: 56368
diff changeset
7 # Copyright (c) 2006-2008, R Oudkerk
76663
25cc0c8fd4e2 Mark multiprocessing files with "Licensed to PSF under a Contributor Agreement"
Richard Oudkerk <shibturn@gmail.com>
parents: 76375
diff changeset
8 # Licensed to PSF under a Contributor Agreement.
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
9 #
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
10
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
11 __all__ = [ 'BaseManager', 'SyncManager', 'BaseProxy', 'Token' ]
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
12
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
13 #
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
14 # Imports
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
15 #
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
16
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
17 import sys
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
18 import threading
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
19 import array
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
20 import queue
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
21
85169
3b82e0d83bf9 Issue #8713: Support alternative start methods in multiprocessing on Unix.
Richard Oudkerk <shibturn@gmail.com>
parents: 84430
diff changeset
22 from time import time as _time
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
23 from traceback import format_exc
85169
3b82e0d83bf9 Issue #8713: Support alternative start methods in multiprocessing on Unix.
Richard Oudkerk <shibturn@gmail.com>
parents: 84430
diff changeset
24
3b82e0d83bf9 Issue #8713: Support alternative start methods in multiprocessing on Unix.
Richard Oudkerk <shibturn@gmail.com>
parents: 84430
diff changeset
25 from . import connection
86387
72a5ac909c7a Issue #18999: Make multiprocessing use context objects.
Richard Oudkerk <shibturn@gmail.com>
parents: 85169
diff changeset
26 from . import context
85169
3b82e0d83bf9 Issue #8713: Support alternative start methods in multiprocessing on Unix.
Richard Oudkerk <shibturn@gmail.com>
parents: 84430
diff changeset
27 from . import pool
3b82e0d83bf9 Issue #8713: Support alternative start methods in multiprocessing on Unix.
Richard Oudkerk <shibturn@gmail.com>
parents: 84430
diff changeset
28 from . import process
3b82e0d83bf9 Issue #8713: Support alternative start methods in multiprocessing on Unix.
Richard Oudkerk <shibturn@gmail.com>
parents: 84430
diff changeset
29 from . import reduction
3b82e0d83bf9 Issue #8713: Support alternative start methods in multiprocessing on Unix.
Richard Oudkerk <shibturn@gmail.com>
parents: 84430
diff changeset
30 from . import util
86387
72a5ac909c7a Issue #18999: Make multiprocessing use context objects.
Richard Oudkerk <shibturn@gmail.com>
parents: 85169
diff changeset
31 from . import get_context
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
32
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
33 #
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
34 # Register some things for pickling
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
35 #
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
36
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
37 def reduce_array(a):
64355
fa8b57f987c5 Issue #8990: array.fromstring() and array.tostring() get renamed to
Antoine Pitrou <solipsis@pitrou.net>
parents: 56368
diff changeset
38 return array.array, (a.typecode, a.tobytes())
85169
3b82e0d83bf9 Issue #8713: Support alternative start methods in multiprocessing on Unix.
Richard Oudkerk <shibturn@gmail.com>
parents: 84430
diff changeset
39 reduction.register(array.array, reduce_array)
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
40
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
41 view_types = [type(getattr({}, name)()) for name in ('items','keys','values')]
48367
d1e864a5c60c make changes for py3k for multiprocessing
Benjamin Peterson <benjamin@python.org>
parents: 48366
diff changeset
42 if view_types[0] is not list: # only needed in Py3.0
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
43 def rebuild_as_list(obj):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
44 return list, (list(obj),)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
45 for view_type in view_types:
85169
3b82e0d83bf9 Issue #8713: Support alternative start methods in multiprocessing on Unix.
Richard Oudkerk <shibturn@gmail.com>
parents: 84430
diff changeset
46 reduction.register(view_type, rebuild_as_list)
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
47
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
48 #
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
49 # Type for identifying shared objects
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
50 #
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
51
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
52 class Token(object):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
53 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
54 Type to uniquely indentify a shared object
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
55 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
56 __slots__ = ('typeid', 'address', 'id')
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
57
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
58 def __init__(self, typeid, address, id):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
59 (self.typeid, self.address, self.id) = (typeid, address, id)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
60
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
61 def __getstate__(self):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
62 return (self.typeid, self.address, self.id)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
63
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
64 def __setstate__(self, state):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
65 (self.typeid, self.address, self.id) = state
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
66
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
67 def __repr__(self):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
68 return 'Token(typeid=%r, address=%r, id=%r)' % \
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
69 (self.typeid, self.address, self.id)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
70
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
71 #
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
72 # Function for communication with a manager's server process
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
73 #
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
74
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
75 def dispatch(c, id, methodname, args=(), kwds={}):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
76 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
77 Send a message to manager using connection `c` and return response
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
78 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
79 c.send((id, methodname, args, kwds))
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
80 kind, result = c.recv()
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
81 if kind == '#RETURN':
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
82 return result
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
83 raise convert_to_error(kind, result)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
84
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
85 def convert_to_error(kind, result):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
86 if kind == '#ERROR':
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
87 return result
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
88 elif kind == '#TRACEBACK':
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
89 assert type(result) is str
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
90 return RemoteError(result)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
91 elif kind == '#UNSERIALIZABLE':
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
92 assert type(result) is str
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
93 return RemoteError('Unserializable message: %s\n' % result)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
94 else:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
95 return ValueError('Unrecognized message type')
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
96
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
97 class RemoteError(Exception):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
98 def __str__(self):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
99 return ('\n' + '-'*75 + '\n' + str(self.args[0]) + '-'*75)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
100
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
101 #
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
102 # Functions for finding the method names of an object
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
103 #
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
104
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
105 def all_methods(obj):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
106 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
107 Return a list of names of methods of `obj`
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
108 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
109 temp = []
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
110 for name in dir(obj):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
111 func = getattr(obj, name)
73165
8e57b5d8f58f Closes #13258: Use callable() built-in in the standard library.
Florent Xicluna <florent.xicluna@gmail.com>
parents: 68257
diff changeset
112 if callable(func):
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
113 temp.append(name)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
114 return temp
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
115
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
116 def public_methods(obj):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
117 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
118 Return a list of names of methods of `obj` which do not start with '_'
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
119 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
120 return [name for name in all_methods(obj) if name[0] != '_']
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
121
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
122 #
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
123 # Server which is run in a process controlled by a manager
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
124 #
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
125
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
126 class Server(object):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
127 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
128 Server class which runs in a process controlled by a manager object
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
129 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
130 public = ['shutdown', 'create', 'accept_connection', 'get_methods',
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
131 'debug_info', 'number_of_objects', 'dummy', 'incref', 'decref']
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
132
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
133 def __init__(self, registry, address, authkey, serializer):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
134 assert isinstance(authkey, bytes)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
135 self.registry = registry
85169
3b82e0d83bf9 Issue #8713: Support alternative start methods in multiprocessing on Unix.
Richard Oudkerk <shibturn@gmail.com>
parents: 84430
diff changeset
136 self.authkey = process.AuthenticationString(authkey)
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
137 Listener, Client = listener_client[serializer]
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
138
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
139 # do authentication later
74144
9b99adef3c78 Issue #13565: Increase multiprocessing's server socket backlog, to avoid
Charles-François Natali <neologix@free.fr>
parents: 73165
diff changeset
140 self.listener = Listener(address=address, backlog=16)
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
141 self.address = self.listener.address
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
142
50962
a99decedf72d merge r68839 forward to py3k
Jesse Noller <jnoller@gmail.com>
parents: 50871
diff changeset
143 self.id_to_obj = {'0': (None, ())}
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
144 self.id_to_refcount = {}
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
145 self.mutex = threading.RLock()
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
146
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
147 def serve_forever(self):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
148 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
149 Run the server forever
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
150 '''
77428
d31e83497c5a Issue #13841: Make child processes exit using sys.exit() on Windows
Richard Oudkerk <shibturn@gmail.com>
parents: 77411
diff changeset
151 self.stop_event = threading.Event()
85169
3b82e0d83bf9 Issue #8713: Support alternative start methods in multiprocessing on Unix.
Richard Oudkerk <shibturn@gmail.com>
parents: 84430
diff changeset
152 process.current_process()._manager_server = self
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
153 try:
77428
d31e83497c5a Issue #13841: Make child processes exit using sys.exit() on Windows
Richard Oudkerk <shibturn@gmail.com>
parents: 77411
diff changeset
154 accepter = threading.Thread(target=self.accepter)
d31e83497c5a Issue #13841: Make child processes exit using sys.exit() on Windows
Richard Oudkerk <shibturn@gmail.com>
parents: 77411
diff changeset
155 accepter.daemon = True
d31e83497c5a Issue #13841: Make child processes exit using sys.exit() on Windows
Richard Oudkerk <shibturn@gmail.com>
parents: 77411
diff changeset
156 accepter.start()
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
157 try:
77428
d31e83497c5a Issue #13841: Make child processes exit using sys.exit() on Windows
Richard Oudkerk <shibturn@gmail.com>
parents: 77411
diff changeset
158 while not self.stop_event.is_set():
d31e83497c5a Issue #13841: Make child processes exit using sys.exit() on Windows
Richard Oudkerk <shibturn@gmail.com>
parents: 77411
diff changeset
159 self.stop_event.wait(1)
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
160 except (KeyboardInterrupt, SystemExit):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
161 pass
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
162 finally:
77428
d31e83497c5a Issue #13841: Make child processes exit using sys.exit() on Windows
Richard Oudkerk <shibturn@gmail.com>
parents: 77411
diff changeset
163 if sys.stdout != sys.__stdout__:
d31e83497c5a Issue #13841: Make child processes exit using sys.exit() on Windows
Richard Oudkerk <shibturn@gmail.com>
parents: 77411
diff changeset
164 util.debug('resetting stdout, stderr')
d31e83497c5a Issue #13841: Make child processes exit using sys.exit() on Windows
Richard Oudkerk <shibturn@gmail.com>
parents: 77411
diff changeset
165 sys.stdout = sys.__stdout__
d31e83497c5a Issue #13841: Make child processes exit using sys.exit() on Windows
Richard Oudkerk <shibturn@gmail.com>
parents: 77411
diff changeset
166 sys.stderr = sys.__stderr__
d31e83497c5a Issue #13841: Make child processes exit using sys.exit() on Windows
Richard Oudkerk <shibturn@gmail.com>
parents: 77411
diff changeset
167 sys.exit(0)
d31e83497c5a Issue #13841: Make child processes exit using sys.exit() on Windows
Richard Oudkerk <shibturn@gmail.com>
parents: 77411
diff changeset
168
d31e83497c5a Issue #13841: Make child processes exit using sys.exit() on Windows
Richard Oudkerk <shibturn@gmail.com>
parents: 77411
diff changeset
169 def accepter(self):
d31e83497c5a Issue #13841: Make child processes exit using sys.exit() on Windows
Richard Oudkerk <shibturn@gmail.com>
parents: 77411
diff changeset
170 while True:
d31e83497c5a Issue #13841: Make child processes exit using sys.exit() on Windows
Richard Oudkerk <shibturn@gmail.com>
parents: 77411
diff changeset
171 try:
d31e83497c5a Issue #13841: Make child processes exit using sys.exit() on Windows
Richard Oudkerk <shibturn@gmail.com>
parents: 77411
diff changeset
172 c = self.listener.accept()
81039
7d69d04522e3 Replace IOError with OSError (#16715)
Andrew Svetlov <andrew.svetlov@gmail.com>
parents: 77511
diff changeset
173 except OSError:
77428
d31e83497c5a Issue #13841: Make child processes exit using sys.exit() on Windows
Richard Oudkerk <shibturn@gmail.com>
parents: 77411
diff changeset
174 continue
d31e83497c5a Issue #13841: Make child processes exit using sys.exit() on Windows
Richard Oudkerk <shibturn@gmail.com>
parents: 77411
diff changeset
175 t = threading.Thread(target=self.handle_request, args=(c,))
d31e83497c5a Issue #13841: Make child processes exit using sys.exit() on Windows
Richard Oudkerk <shibturn@gmail.com>
parents: 77411
diff changeset
176 t.daemon = True
d31e83497c5a Issue #13841: Make child processes exit using sys.exit() on Windows
Richard Oudkerk <shibturn@gmail.com>
parents: 77411
diff changeset
177 t.start()
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
178
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
179 def handle_request(self, c):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
180 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
181 Handle a new connection
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
182 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
183 funcname = result = request = None
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
184 try:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
185 connection.deliver_challenge(c, self.authkey)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
186 connection.answer_challenge(c, self.authkey)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
187 request = c.recv()
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
188 ignore, funcname, args, kwds = request
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
189 assert funcname in self.public, '%r unrecognized' % funcname
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
190 func = getattr(self, funcname)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
191 except Exception:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
192 msg = ('#TRACEBACK', format_exc())
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
193 else:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
194 try:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
195 result = func(c, *args, **kwds)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
196 except Exception:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
197 msg = ('#TRACEBACK', format_exc())
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
198 else:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
199 msg = ('#RETURN', result)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
200 try:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
201 c.send(msg)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
202 except Exception as e:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
203 try:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
204 c.send(('#TRACEBACK', format_exc()))
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
205 except Exception:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
206 pass
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
207 util.info('Failure to send message: %r', msg)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
208 util.info(' ... request was %r', request)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
209 util.info(' ... exception was %r', e)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
210
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
211 c.close()
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
212
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
213 def serve_client(self, conn):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
214 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
215 Handle requests from the proxies in a particular process/thread
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
216 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
217 util.debug('starting server thread to service %r',
48747
1fdfa382ae13 Merged revisions 65824 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 48367
diff changeset
218 threading.current_thread().name)
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
219
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
220 recv = conn.recv
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
221 send = conn.send
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
222 id_to_obj = self.id_to_obj
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
223
77428
d31e83497c5a Issue #13841: Make child processes exit using sys.exit() on Windows
Richard Oudkerk <shibturn@gmail.com>
parents: 77411
diff changeset
224 while not self.stop_event.is_set():
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
225
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
226 try:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
227 methodname = obj = None
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
228 request = recv()
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
229 ident, methodname, args, kwds = request
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
230 obj, exposed, gettypeid = id_to_obj[ident]
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
231
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
232 if methodname not in exposed:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
233 raise AttributeError(
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
234 'method %r of %r object is not in exposed=%r' %
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
235 (methodname, type(obj), exposed)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
236 )
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
237
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
238 function = getattr(obj, methodname)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
239
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
240 try:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
241 res = function(*args, **kwds)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
242 except Exception as e:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
243 msg = ('#ERROR', e)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
244 else:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
245 typeid = gettypeid and gettypeid.get(methodname, None)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
246 if typeid:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
247 rident, rexposed = self.create(conn, typeid, res)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
248 token = Token(typeid, self.address, rident)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
249 msg = ('#PROXY', (rexposed, token))
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
250 else:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
251 msg = ('#RETURN', res)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
252
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
253 except AttributeError:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
254 if methodname is None:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
255 msg = ('#TRACEBACK', format_exc())
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
256 else:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
257 try:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
258 fallback_func = self.fallback_mapping[methodname]
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
259 result = fallback_func(
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
260 self, conn, ident, obj, *args, **kwds
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
261 )
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
262 msg = ('#RETURN', result)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
263 except Exception:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
264 msg = ('#TRACEBACK', format_exc())
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
265
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
266 except EOFError:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
267 util.debug('got EOF -- exiting thread serving %r',
48747
1fdfa382ae13 Merged revisions 65824 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 48367
diff changeset
268 threading.current_thread().name)
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
269 sys.exit(0)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
270
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
271 except Exception:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
272 msg = ('#TRACEBACK', format_exc())
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
273
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
274 try:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
275 try:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
276 send(msg)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
277 except Exception as e:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
278 send(('#UNSERIALIZABLE', repr(msg)))
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
279 except Exception as e:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
280 util.info('exception in thread serving %r',
48747
1fdfa382ae13 Merged revisions 65824 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 48367
diff changeset
281 threading.current_thread().name)
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
282 util.info(' ... message was %r', msg)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
283 util.info(' ... exception was %r', e)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
284 conn.close()
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
285 sys.exit(1)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
286
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
287 def fallback_getvalue(self, conn, ident, obj):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
288 return obj
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
289
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
290 def fallback_str(self, conn, ident, obj):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
291 return str(obj)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
292
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
293 def fallback_repr(self, conn, ident, obj):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
294 return repr(obj)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
295
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
296 fallback_mapping = {
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
297 '__str__':fallback_str,
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
298 '__repr__':fallback_repr,
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
299 '#GETVALUE':fallback_getvalue
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
300 }
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
301
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
302 def dummy(self, c):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
303 pass
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
304
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
305 def debug_info(self, c):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
306 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
307 Return some info --- useful to spot problems with refcounting
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
308 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
309 self.mutex.acquire()
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
310 try:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
311 result = []
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
312 keys = list(self.id_to_obj.keys())
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
313 keys.sort()
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
314 for ident in keys:
50962
a99decedf72d merge r68839 forward to py3k
Jesse Noller <jnoller@gmail.com>
parents: 50871
diff changeset
315 if ident != '0':
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
316 result.append(' %s: refcount=%s\n %s' %
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
317 (ident, self.id_to_refcount[ident],
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
318 str(self.id_to_obj[ident][0])[:75]))
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
319 return '\n'.join(result)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
320 finally:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
321 self.mutex.release()
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
322
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
323 def number_of_objects(self, c):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
324 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
325 Number of shared objects
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
326 '''
50962
a99decedf72d merge r68839 forward to py3k
Jesse Noller <jnoller@gmail.com>
parents: 50871
diff changeset
327 return len(self.id_to_obj) - 1 # don't count ident='0'
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
328
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
329 def shutdown(self, c):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
330 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
331 Shutdown this process
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
332 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
333 try:
77428
d31e83497c5a Issue #13841: Make child processes exit using sys.exit() on Windows
Richard Oudkerk <shibturn@gmail.com>
parents: 77411
diff changeset
334 util.debug('manager received shutdown message')
d31e83497c5a Issue #13841: Make child processes exit using sys.exit() on Windows
Richard Oudkerk <shibturn@gmail.com>
parents: 77411
diff changeset
335 c.send(('#RETURN', None))
d31e83497c5a Issue #13841: Make child processes exit using sys.exit() on Windows
Richard Oudkerk <shibturn@gmail.com>
parents: 77411
diff changeset
336 except:
d31e83497c5a Issue #13841: Make child processes exit using sys.exit() on Windows
Richard Oudkerk <shibturn@gmail.com>
parents: 77411
diff changeset
337 import traceback
d31e83497c5a Issue #13841: Make child processes exit using sys.exit() on Windows
Richard Oudkerk <shibturn@gmail.com>
parents: 77411
diff changeset
338 traceback.print_exc()
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
339 finally:
77428
d31e83497c5a Issue #13841: Make child processes exit using sys.exit() on Windows
Richard Oudkerk <shibturn@gmail.com>
parents: 77411
diff changeset
340 self.stop_event.set()
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
341
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
342 def create(self, c, typeid, *args, **kwds):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
343 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
344 Create a new shared object and return its id
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
345 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
346 self.mutex.acquire()
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
347 try:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
348 callable, exposed, method_to_typeid, proxytype = \
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
349 self.registry[typeid]
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
350
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
351 if callable is None:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
352 assert len(args) == 1 and not kwds
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
353 obj = args[0]
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
354 else:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
355 obj = callable(*args, **kwds)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
356
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
357 if exposed is None:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
358 exposed = public_methods(obj)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
359 if method_to_typeid is not None:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
360 assert type(method_to_typeid) is dict
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
361 exposed = list(exposed) + list(method_to_typeid)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
362
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
363 ident = '%x' % id(obj) # convert to string because xmlrpclib
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
364 # only has 32 bit signed integers
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
365 util.debug('%r callable returned object with id %r', typeid, ident)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
366
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
367 self.id_to_obj[ident] = (obj, set(exposed), method_to_typeid)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
368 if ident not in self.id_to_refcount:
48966
750da2d18089 Merge r66115 forward to py3k, resolves issue3419
Jesse Noller <jnoller@gmail.com>
parents: 48792
diff changeset
369 self.id_to_refcount[ident] = 0
750da2d18089 Merge r66115 forward to py3k, resolves issue3419
Jesse Noller <jnoller@gmail.com>
parents: 48792
diff changeset
370 # increment the reference count immediately, to avoid
750da2d18089 Merge r66115 forward to py3k, resolves issue3419
Jesse Noller <jnoller@gmail.com>
parents: 48792
diff changeset
371 # this object being garbage collected before a Proxy
750da2d18089 Merge r66115 forward to py3k, resolves issue3419
Jesse Noller <jnoller@gmail.com>
parents: 48792
diff changeset
372 # object for it can be created. The caller of create()
750da2d18089 Merge r66115 forward to py3k, resolves issue3419
Jesse Noller <jnoller@gmail.com>
parents: 48792
diff changeset
373 # is responsible for doing a decref once the Proxy object
750da2d18089 Merge r66115 forward to py3k, resolves issue3419
Jesse Noller <jnoller@gmail.com>
parents: 48792
diff changeset
374 # has been created.
750da2d18089 Merge r66115 forward to py3k, resolves issue3419
Jesse Noller <jnoller@gmail.com>
parents: 48792
diff changeset
375 self.incref(c, ident)
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
376 return ident, tuple(exposed)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
377 finally:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
378 self.mutex.release()
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
379
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
380 def get_methods(self, c, token):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
381 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
382 Return the methods of the shared object indicated by token
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
383 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
384 return tuple(self.id_to_obj[token.id][1])
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
385
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
386 def accept_connection(self, c, name):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
387 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
388 Spawn a new thread to serve this connection
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
389 '''
48747
1fdfa382ae13 Merged revisions 65824 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 48367
diff changeset
390 threading.current_thread().name = name
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
391 c.send(('#RETURN', None))
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
392 self.serve_client(c)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
393
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
394 def incref(self, c, ident):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
395 self.mutex.acquire()
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
396 try:
48966
750da2d18089 Merge r66115 forward to py3k, resolves issue3419
Jesse Noller <jnoller@gmail.com>
parents: 48792
diff changeset
397 self.id_to_refcount[ident] += 1
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
398 finally:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
399 self.mutex.release()
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
400
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
401 def decref(self, c, ident):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
402 self.mutex.acquire()
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
403 try:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
404 assert self.id_to_refcount[ident] >= 1
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
405 self.id_to_refcount[ident] -= 1
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
406 if self.id_to_refcount[ident] == 0:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
407 del self.id_to_obj[ident], self.id_to_refcount[ident]
56368
c9bd62efe6ce Merged revisions 75231 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 53077
diff changeset
408 util.debug('disposing of obj with id %r', ident)
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
409 finally:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
410 self.mutex.release()
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
411
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
412 #
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
413 # Class to represent state of a manager
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
414 #
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
415
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
416 class State(object):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
417 __slots__ = ['value']
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
418 INITIAL = 0
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
419 STARTED = 1
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
420 SHUTDOWN = 2
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
421
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
422 #
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
423 # Mapping from serializer name to Listener and Client types
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
424 #
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
425
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
426 listener_client = {
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
427 'pickle' : (connection.Listener, connection.Client),
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
428 'xmlrpclib' : (connection.XmlListener, connection.XmlClient)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
429 }
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
430
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
431 #
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
432 # Definition of BaseManager
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
433 #
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
434
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
435 class BaseManager(object):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
436 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
437 Base class for managers
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
438 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
439 _registry = {}
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
440 _Server = Server
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
441
86387
72a5ac909c7a Issue #18999: Make multiprocessing use context objects.
Richard Oudkerk <shibturn@gmail.com>
parents: 85169
diff changeset
442 def __init__(self, address=None, authkey=None, serializer='pickle',
72a5ac909c7a Issue #18999: Make multiprocessing use context objects.
Richard Oudkerk <shibturn@gmail.com>
parents: 85169
diff changeset
443 ctx=None):
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
444 if authkey is None:
85169
3b82e0d83bf9 Issue #8713: Support alternative start methods in multiprocessing on Unix.
Richard Oudkerk <shibturn@gmail.com>
parents: 84430
diff changeset
445 authkey = process.current_process().authkey
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
446 self._address = address # XXX not final address if eg ('', 0)
85169
3b82e0d83bf9 Issue #8713: Support alternative start methods in multiprocessing on Unix.
Richard Oudkerk <shibturn@gmail.com>
parents: 84430
diff changeset
447 self._authkey = process.AuthenticationString(authkey)
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
448 self._state = State()
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
449 self._state.value = State.INITIAL
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
450 self._serializer = serializer
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
451 self._Listener, self._Client = listener_client[serializer]
86387
72a5ac909c7a Issue #18999: Make multiprocessing use context objects.
Richard Oudkerk <shibturn@gmail.com>
parents: 85169
diff changeset
452 self._ctx = ctx or get_context()
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
453
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
454 def get_server(self):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
455 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
456 Return server object with serve_forever() method and address attribute
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
457 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
458 assert self._state.value == State.INITIAL
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
459 return Server(self._registry, self._address,
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
460 self._authkey, self._serializer)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
461
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
462 def connect(self):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
463 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
464 Connect manager object to the server process
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
465 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
466 Listener, Client = listener_client[self._serializer]
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
467 conn = Client(self._address, authkey=self._authkey)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
468 dispatch(conn, None, 'dummy')
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
469 self._state.value = State.STARTED
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
470
53077
48485604ed48 Merged revisions 70912,70944,70968,71033,71041,71208,71263,71286,71395-71396,71405-71406,71485,71492,71494 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 50962
diff changeset
471 def start(self, initializer=None, initargs=()):
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
472 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
473 Spawn a server process for this manager object
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
474 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
475 assert self._state.value == State.INITIAL
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
476
73165
8e57b5d8f58f Closes #13258: Use callable() built-in in the standard library.
Florent Xicluna <florent.xicluna@gmail.com>
parents: 68257
diff changeset
477 if initializer is not None and not callable(initializer):
53077
48485604ed48 Merged revisions 70912,70944,70968,71033,71041,71208,71263,71286,71395-71396,71405-71406,71485,71492,71494 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 50962
diff changeset
478 raise TypeError('initializer must be a callable')
48485604ed48 Merged revisions 70912,70944,70968,71033,71041,71208,71263,71286,71395-71396,71405-71406,71485,71492,71494 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 50962
diff changeset
479
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
480 # pipe over which we will retrieve address of server
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
481 reader, writer = connection.Pipe(duplex=False)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
482
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
483 # spawn process which runs a server
86387
72a5ac909c7a Issue #18999: Make multiprocessing use context objects.
Richard Oudkerk <shibturn@gmail.com>
parents: 85169
diff changeset
484 self._process = self._ctx.Process(
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
485 target=type(self)._run_server,
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
486 args=(self._registry, self._address, self._authkey,
53077
48485604ed48 Merged revisions 70912,70944,70968,71033,71041,71208,71263,71286,71395-71396,71405-71406,71485,71492,71494 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 50962
diff changeset
487 self._serializer, writer, initializer, initargs),
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
488 )
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
489 ident = ':'.join(str(i) for i in self._process._identity)
48773
25f06a703f89 Merged revisions 65864 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 48749
diff changeset
490 self._process.name = type(self).__name__ + '-' + ident
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
491 self._process.start()
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
492
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
493 # get address of server
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
494 writer.close()
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
495 self._address = reader.recv()
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
496 reader.close()
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
497
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
498 # register a finalizer
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
499 self._state.value = State.STARTED
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
500 self.shutdown = util.Finalize(
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
501 self, type(self)._finalize_manager,
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
502 args=(self._process, self._address, self._authkey,
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
503 self._state, self._Client),
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
504 exitpriority=0
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
505 )
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
506
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
507 @classmethod
53077
48485604ed48 Merged revisions 70912,70944,70968,71033,71041,71208,71263,71286,71395-71396,71405-71406,71485,71492,71494 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 50962
diff changeset
508 def _run_server(cls, registry, address, authkey, serializer, writer,
48485604ed48 Merged revisions 70912,70944,70968,71033,71041,71208,71263,71286,71395-71396,71405-71406,71485,71492,71494 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 50962
diff changeset
509 initializer=None, initargs=()):
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
510 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
511 Create a server, report its address and run it
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
512 '''
53077
48485604ed48 Merged revisions 70912,70944,70968,71033,71041,71208,71263,71286,71395-71396,71405-71406,71485,71492,71494 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 50962
diff changeset
513 if initializer is not None:
48485604ed48 Merged revisions 70912,70944,70968,71033,71041,71208,71263,71286,71395-71396,71405-71406,71485,71492,71494 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 50962
diff changeset
514 initializer(*initargs)
48485604ed48 Merged revisions 70912,70944,70968,71033,71041,71208,71263,71286,71395-71396,71405-71406,71485,71492,71494 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 50962
diff changeset
515
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
516 # create server
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
517 server = cls._Server(registry, address, authkey, serializer)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
518
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
519 # inform parent process of the server's address
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
520 writer.send(server.address)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
521 writer.close()
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
522
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
523 # run the manager
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
524 util.info('manager serving at %r', server.address)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
525 server.serve_forever()
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
526
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
527 def _create(self, typeid, *args, **kwds):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
528 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
529 Create a new shared object; return the token and exposed tuple
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
530 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
531 assert self._state.value == State.STARTED, 'server not yet started'
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
532 conn = self._Client(self._address, authkey=self._authkey)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
533 try:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
534 id, exposed = dispatch(conn, None, 'create', (typeid,)+args, kwds)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
535 finally:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
536 conn.close()
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
537 return Token(typeid, self._address, id), exposed
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
538
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
539 def join(self, timeout=None):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
540 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
541 Join the manager process (if it has been spawned)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
542 '''
76734
5a4276fdf4b7 Fix dangling warning for test_multiprocessing
Richard Oudkerk <shibturn@gmail.com>
parents: 76663
diff changeset
543 if self._process is not None:
5a4276fdf4b7 Fix dangling warning for test_multiprocessing
Richard Oudkerk <shibturn@gmail.com>
parents: 76663
diff changeset
544 self._process.join(timeout)
5a4276fdf4b7 Fix dangling warning for test_multiprocessing
Richard Oudkerk <shibturn@gmail.com>
parents: 76663
diff changeset
545 if not self._process.is_alive():
5a4276fdf4b7 Fix dangling warning for test_multiprocessing
Richard Oudkerk <shibturn@gmail.com>
parents: 76663
diff changeset
546 self._process = None
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
547
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
548 def _debug_info(self):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
549 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
550 Return some info about the servers shared objects and connections
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
551 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
552 conn = self._Client(self._address, authkey=self._authkey)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
553 try:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
554 return dispatch(conn, None, 'debug_info')
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
555 finally:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
556 conn.close()
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
557
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
558 def _number_of_objects(self):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
559 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
560 Return the number of shared objects
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
561 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
562 conn = self._Client(self._address, authkey=self._authkey)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
563 try:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
564 return dispatch(conn, None, 'number_of_objects')
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
565 finally:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
566 conn.close()
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
567
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
568 def __enter__(self):
77511
198382b4bcd0 Issue #15064: Make BaseManager.__enter__() start server if necessary.
Richard Oudkerk <shibturn@gmail.com>
parents: 77447
diff changeset
569 if self._state.value == State.INITIAL:
198382b4bcd0 Issue #15064: Make BaseManager.__enter__() start server if necessary.
Richard Oudkerk <shibturn@gmail.com>
parents: 77447
diff changeset
570 self.start()
198382b4bcd0 Issue #15064: Make BaseManager.__enter__() start server if necessary.
Richard Oudkerk <shibturn@gmail.com>
parents: 77447
diff changeset
571 assert self._state.value == State.STARTED
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
572 return self
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
573
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
574 def __exit__(self, exc_type, exc_val, exc_tb):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
575 self.shutdown()
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
576
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
577 @staticmethod
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
578 def _finalize_manager(process, address, authkey, state, _Client):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
579 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
580 Shutdown the manager process; will be registered as a finalizer
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
581 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
582 if process.is_alive():
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
583 util.info('sending shutdown message to manager')
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
584 try:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
585 conn = _Client(address, authkey=authkey)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
586 try:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
587 dispatch(conn, None, 'shutdown')
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
588 finally:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
589 conn.close()
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
590 except Exception:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
591 pass
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
592
77447
831ae71d0bdc Increase timeout used when waiting for manager to shutdown cleanly
Richard Oudkerk <shibturn@gmail.com>
parents: 77442
diff changeset
593 process.join(timeout=1.0)
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
594 if process.is_alive():
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
595 util.info('manager still alive')
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
596 if hasattr(process, 'terminate'):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
597 util.info('trying to `terminate()` manager process')
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
598 process.terminate()
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
599 process.join(timeout=0.1)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
600 if process.is_alive():
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
601 util.info('manager still alive after terminate')
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
602
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
603 state.value = State.SHUTDOWN
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
604 try:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
605 del BaseProxy._address_to_local[address]
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
606 except KeyError:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
607 pass
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
608
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
609 address = property(lambda self: self._address)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
610
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
611 @classmethod
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
612 def register(cls, typeid, callable=None, proxytype=None, exposed=None,
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
613 method_to_typeid=None, create_method=True):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
614 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
615 Register a typeid with the manager type
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
616 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
617 if '_registry' not in cls.__dict__:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
618 cls._registry = cls._registry.copy()
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
619
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
620 if proxytype is None:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
621 proxytype = AutoProxy
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
622
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
623 exposed = exposed or getattr(proxytype, '_exposed_', None)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
624
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
625 method_to_typeid = method_to_typeid or \
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
626 getattr(proxytype, '_method_to_typeid_', None)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
627
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
628 if method_to_typeid:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
629 for key, value in list(method_to_typeid.items()):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
630 assert type(key) is str, '%r is not a string' % key
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
631 assert type(value) is str, '%r is not a string' % value
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
632
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
633 cls._registry[typeid] = (
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
634 callable, exposed, method_to_typeid, proxytype
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
635 )
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
636
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
637 if create_method:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
638 def temp(self, *args, **kwds):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
639 util.debug('requesting creation of a shared %r object', typeid)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
640 token, exp = self._create(typeid, *args, **kwds)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
641 proxy = proxytype(
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
642 token, self._serializer, manager=self,
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
643 authkey=self._authkey, exposed=exp
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
644 )
48966
750da2d18089 Merge r66115 forward to py3k, resolves issue3419
Jesse Noller <jnoller@gmail.com>
parents: 48792
diff changeset
645 conn = self._Client(token.address, authkey=self._authkey)
750da2d18089 Merge r66115 forward to py3k, resolves issue3419
Jesse Noller <jnoller@gmail.com>
parents: 48792
diff changeset
646 dispatch(conn, None, 'decref', (token.id,))
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
647 return proxy
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
648 temp.__name__ = typeid
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
649 setattr(cls, typeid, temp)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
650
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
651 #
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
652 # Subclass of set which get cleared after a fork
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
653 #
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
654
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
655 class ProcessLocalSet(set):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
656 def __init__(self):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
657 util.register_after_fork(self, lambda obj: obj.clear())
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
658 def __reduce__(self):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
659 return type(self), ()
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
660
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
661 #
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
662 # Definition of BaseProxy
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
663 #
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
664
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
665 class BaseProxy(object):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
666 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
667 A base for proxies of shared objects
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
668 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
669 _address_to_local = {}
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
670 _mutex = util.ForkAwareThreadLock()
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
671
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
672 def __init__(self, token, serializer, manager=None,
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
673 authkey=None, exposed=None, incref=True):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
674 BaseProxy._mutex.acquire()
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
675 try:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
676 tls_idset = BaseProxy._address_to_local.get(token.address, None)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
677 if tls_idset is None:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
678 tls_idset = util.ForkAwareLocal(), ProcessLocalSet()
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
679 BaseProxy._address_to_local[token.address] = tls_idset
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
680 finally:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
681 BaseProxy._mutex.release()
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
682
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
683 # self._tls is used to record the connection used by this
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
684 # thread to communicate with the manager at token.address
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
685 self._tls = tls_idset[0]
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
686
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
687 # self._idset is used to record the identities of all shared
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
688 # objects for which the current process owns references and
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
689 # which are in the manager at token.address
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
690 self._idset = tls_idset[1]
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
691
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
692 self._token = token
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
693 self._id = self._token.id
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
694 self._manager = manager
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
695 self._serializer = serializer
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
696 self._Client = listener_client[serializer][1]
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
697
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
698 if authkey is not None:
85169
3b82e0d83bf9 Issue #8713: Support alternative start methods in multiprocessing on Unix.
Richard Oudkerk <shibturn@gmail.com>
parents: 84430
diff changeset
699 self._authkey = process.AuthenticationString(authkey)
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
700 elif self._manager is not None:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
701 self._authkey = self._manager._authkey
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
702 else:
85169
3b82e0d83bf9 Issue #8713: Support alternative start methods in multiprocessing on Unix.
Richard Oudkerk <shibturn@gmail.com>
parents: 84430
diff changeset
703 self._authkey = process.current_process().authkey
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
704
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
705 if incref:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
706 self._incref()
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
707
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
708 util.register_after_fork(self, BaseProxy._after_fork)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
709
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
710 def _connect(self):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
711 util.debug('making connection to manager')
85169
3b82e0d83bf9 Issue #8713: Support alternative start methods in multiprocessing on Unix.
Richard Oudkerk <shibturn@gmail.com>
parents: 84430
diff changeset
712 name = process.current_process().name
48747
1fdfa382ae13 Merged revisions 65824 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 48367
diff changeset
713 if threading.current_thread().name != 'MainThread':
1fdfa382ae13 Merged revisions 65824 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 48367
diff changeset
714 name += '|' + threading.current_thread().name
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
715 conn = self._Client(self._token.address, authkey=self._authkey)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
716 dispatch(conn, None, 'accept_connection', (name,))
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
717 self._tls.connection = conn
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
718
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
719 def _callmethod(self, methodname, args=(), kwds={}):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
720 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
721 Try to call a method of the referrent and return a copy of the result
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
722 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
723 try:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
724 conn = self._tls.connection
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
725 except AttributeError:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
726 util.debug('thread %r does not own a connection',
48747
1fdfa382ae13 Merged revisions 65824 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 48367
diff changeset
727 threading.current_thread().name)
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
728 self._connect()
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
729 conn = self._tls.connection
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
730
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
731 conn.send((self._id, methodname, args, kwds))
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
732 kind, result = conn.recv()
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
733
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
734 if kind == '#RETURN':
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
735 return result
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
736 elif kind == '#PROXY':
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
737 exposed, token = result
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
738 proxytype = self._manager._registry[token.typeid][-1]
84429
be4b9e677187 Issue #17261: Ensure multiprocessing's proxies use proper address.
Richard Oudkerk <shibturn@gmail.com>
parents: 77511
diff changeset
739 token.address = self._token.address
48966
750da2d18089 Merge r66115 forward to py3k, resolves issue3419
Jesse Noller <jnoller@gmail.com>
parents: 48792
diff changeset
740 proxy = proxytype(
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
741 token, self._serializer, manager=self._manager,
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
742 authkey=self._authkey, exposed=exposed
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
743 )
48966
750da2d18089 Merge r66115 forward to py3k, resolves issue3419
Jesse Noller <jnoller@gmail.com>
parents: 48792
diff changeset
744 conn = self._Client(token.address, authkey=self._authkey)
750da2d18089 Merge r66115 forward to py3k, resolves issue3419
Jesse Noller <jnoller@gmail.com>
parents: 48792
diff changeset
745 dispatch(conn, None, 'decref', (token.id,))
750da2d18089 Merge r66115 forward to py3k, resolves issue3419
Jesse Noller <jnoller@gmail.com>
parents: 48792
diff changeset
746 return proxy
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
747 raise convert_to_error(kind, result)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
748
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
749 def _getvalue(self):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
750 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
751 Get a copy of the value of the referent
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
752 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
753 return self._callmethod('#GETVALUE')
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
754
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
755 def _incref(self):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
756 conn = self._Client(self._token.address, authkey=self._authkey)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
757 dispatch(conn, None, 'incref', (self._id,))
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
758 util.debug('INCREF %r', self._token.id)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
759
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
760 self._idset.add(self._id)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
761
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
762 state = self._manager and self._manager._state
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
763
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
764 self._close = util.Finalize(
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
765 self, BaseProxy._decref,
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
766 args=(self._token, self._authkey, state,
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
767 self._tls, self._idset, self._Client),
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
768 exitpriority=10
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
769 )
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
770
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
771 @staticmethod
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
772 def _decref(token, authkey, state, tls, idset, _Client):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
773 idset.discard(token.id)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
774
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
775 # check whether manager is still alive
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
776 if state is None or state.value == State.STARTED:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
777 # tell manager this process no longer cares about referent
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
778 try:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
779 util.debug('DECREF %r', token.id)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
780 conn = _Client(token.address, authkey=authkey)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
781 dispatch(conn, None, 'decref', (token.id,))
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
782 except Exception as e:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
783 util.debug('... decref failed %s', e)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
784
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
785 else:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
786 util.debug('DECREF %r -- manager already shutdown', token.id)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
787
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
788 # check whether we can close this thread's connection because
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
789 # the process owns no more references to objects for this manager
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
790 if not idset and hasattr(tls, 'connection'):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
791 util.debug('thread %r has no more proxies so closing conn',
48747
1fdfa382ae13 Merged revisions 65824 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 48367
diff changeset
792 threading.current_thread().name)
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
793 tls.connection.close()
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
794 del tls.connection
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
795
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
796 def _after_fork(self):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
797 self._manager = None
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
798 try:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
799 self._incref()
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
800 except Exception as e:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
801 # the proxy may just be for a manager which has shutdown
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
802 util.info('incref failed: %s' % e)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
803
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
804 def __reduce__(self):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
805 kwds = {}
86387
72a5ac909c7a Issue #18999: Make multiprocessing use context objects.
Richard Oudkerk <shibturn@gmail.com>
parents: 85169
diff changeset
806 if context.get_spawning_popen() is not None:
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
807 kwds['authkey'] = self._authkey
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
808
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
809 if getattr(self, '_isauto', False):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
810 kwds['exposed'] = self._exposed_
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
811 return (RebuildProxy,
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
812 (AutoProxy, self._token, self._serializer, kwds))
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
813 else:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
814 return (RebuildProxy,
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
815 (type(self), self._token, self._serializer, kwds))
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
816
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
817 def __deepcopy__(self, memo):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
818 return self._getvalue()
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
819
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
820 def __repr__(self):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
821 return '<%s object, typeid %r at %s>' % \
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
822 (type(self).__name__, self._token.typeid, '0x%x' % id(self))
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
823
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
824 def __str__(self):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
825 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
826 Return representation of the referent (or a fall-back if that fails)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
827 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
828 try:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
829 return self._callmethod('__repr__')
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
830 except Exception:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
831 return repr(self)[:-1] + "; '__str__()' failed>"
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
832
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
833 #
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
834 # Function used for unpickling
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
835 #
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
836
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
837 def RebuildProxy(func, token, serializer, kwds):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
838 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
839 Function used for unpickling proxy objects.
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
840
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
841 If possible the shared object is returned, or otherwise a proxy for it.
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
842 '''
85169
3b82e0d83bf9 Issue #8713: Support alternative start methods in multiprocessing on Unix.
Richard Oudkerk <shibturn@gmail.com>
parents: 84430
diff changeset
843 server = getattr(process.current_process(), '_manager_server', None)
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
844
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
845 if server and server.address == token.address:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
846 return server.id_to_obj[token.id][0]
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
847 else:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
848 incref = (
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
849 kwds.pop('incref', True) and
85169
3b82e0d83bf9 Issue #8713: Support alternative start methods in multiprocessing on Unix.
Richard Oudkerk <shibturn@gmail.com>
parents: 84430
diff changeset
850 not getattr(process.current_process(), '_inheriting', False)
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
851 )
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
852 return func(token, serializer, incref=incref, **kwds)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
853
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
854 #
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
855 # Functions to create proxies and proxy types
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
856 #
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
857
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
858 def MakeProxyType(name, exposed, _cache={}):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
859 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
860 Return an proxy type whose methods are given by `exposed`
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
861 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
862 exposed = tuple(exposed)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
863 try:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
864 return _cache[(name, exposed)]
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
865 except KeyError:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
866 pass
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
867
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
868 dic = {}
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
869
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
870 for meth in exposed:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
871 exec('''def %s(self, *args, **kwds):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
872 return self._callmethod(%r, args, kwds)''' % (meth, meth), dic)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
873
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
874 ProxyType = type(name, (BaseProxy,), dic)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
875 ProxyType._exposed_ = exposed
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
876 _cache[(name, exposed)] = ProxyType
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
877 return ProxyType
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
878
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
879
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
880 def AutoProxy(token, serializer, manager=None, authkey=None,
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
881 exposed=None, incref=True):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
882 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
883 Return an auto-proxy for `token`
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
884 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
885 _Client = listener_client[serializer][1]
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
886
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
887 if exposed is None:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
888 conn = _Client(token.address, authkey=authkey)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
889 try:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
890 exposed = dispatch(conn, None, 'get_methods', (token,))
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
891 finally:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
892 conn.close()
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
893
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
894 if authkey is None and manager is not None:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
895 authkey = manager._authkey
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
896 if authkey is None:
85169
3b82e0d83bf9 Issue #8713: Support alternative start methods in multiprocessing on Unix.
Richard Oudkerk <shibturn@gmail.com>
parents: 84430
diff changeset
897 authkey = process.current_process().authkey
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
898
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
899 ProxyType = MakeProxyType('AutoProxy[%s]' % token.typeid, exposed)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
900 proxy = ProxyType(token, serializer, manager=manager, authkey=authkey,
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
901 incref=incref)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
902 proxy._isauto = True
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
903 return proxy
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
904
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
905 #
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
906 # Types/callables which we will register with SyncManager
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
907 #
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
908
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
909 class Namespace(object):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
910 def __init__(self, **kwds):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
911 self.__dict__.update(kwds)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
912 def __repr__(self):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
913 items = list(self.__dict__.items())
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
914 temp = []
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
915 for name, value in items:
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
916 if not name.startswith('_'):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
917 temp.append('%s=%r' % (name, value))
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
918 temp.sort()
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
919 return 'Namespace(%s)' % str.join(', ', temp)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
920
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
921 class Value(object):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
922 def __init__(self, typecode, value, lock=True):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
923 self._typecode = typecode
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
924 self._value = value
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
925 def get(self):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
926 return self._value
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
927 def set(self, value):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
928 self._value = value
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
929 def __repr__(self):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
930 return '%s(%r, %r)'%(type(self).__name__, self._typecode, self._value)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
931 value = property(get, set)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
932
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
933 def Array(typecode, sequence, lock=True):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
934 return array.array(typecode, sequence)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
935
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
936 #
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
937 # Proxy types used by SyncManager
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
938 #
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
939
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
940 class IteratorProxy(BaseProxy):
48367
d1e864a5c60c make changes for py3k for multiprocessing
Benjamin Peterson <benjamin@python.org>
parents: 48366
diff changeset
941 _exposed_ = ('__next__', 'send', 'throw', 'close')
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
942 def __iter__(self):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
943 return self
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
944 def __next__(self, *args):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
945 return self._callmethod('__next__', args)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
946 def send(self, *args):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
947 return self._callmethod('send', args)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
948 def throw(self, *args):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
949 return self._callmethod('throw', args)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
950 def close(self, *args):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
951 return self._callmethod('close', args)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
952
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
953
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
954 class AcquirerProxy(BaseProxy):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
955 _exposed_ = ('acquire', 'release')
76800
b4a1d9287780 Make AcquirerProxy.acquire() support timeout argument
Richard Oudkerk <shibturn@gmail.com>
parents: 76734
diff changeset
956 def acquire(self, blocking=True, timeout=None):
b4a1d9287780 Make AcquirerProxy.acquire() support timeout argument
Richard Oudkerk <shibturn@gmail.com>
parents: 76734
diff changeset
957 args = (blocking,) if timeout is None else (blocking, timeout)
b4a1d9287780 Make AcquirerProxy.acquire() support timeout argument
Richard Oudkerk <shibturn@gmail.com>
parents: 76734
diff changeset
958 return self._callmethod('acquire', args)
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
959 def release(self):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
960 return self._callmethod('release')
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
961 def __enter__(self):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
962 return self._callmethod('acquire')
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
963 def __exit__(self, exc_type, exc_val, exc_tb):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
964 return self._callmethod('release')
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
965
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
966
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
967 class ConditionProxy(AcquirerProxy):
47615
045f83a1b864 Merged revisions 64125 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 47595
diff changeset
968 _exposed_ = ('acquire', 'release', 'wait', 'notify', 'notify_all')
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
969 def wait(self, timeout=None):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
970 return self._callmethod('wait', (timeout,))
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
971 def notify(self):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
972 return self._callmethod('notify')
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
973 def notify_all(self):
47615
045f83a1b864 Merged revisions 64125 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 47595
diff changeset
974 return self._callmethod('notify_all')
76375
5606ee052783 Issue #14087: multiprocessing: add Condition.wait_for(). Patch by sbt.
Charles-François Natali <neologix@free.fr>
parents: 74145
diff changeset
975 def wait_for(self, predicate, timeout=None):
5606ee052783 Issue #14087: multiprocessing: add Condition.wait_for(). Patch by sbt.
Charles-François Natali <neologix@free.fr>
parents: 74145
diff changeset
976 result = predicate()
5606ee052783 Issue #14087: multiprocessing: add Condition.wait_for(). Patch by sbt.
Charles-François Natali <neologix@free.fr>
parents: 74145
diff changeset
977 if result:
5606ee052783 Issue #14087: multiprocessing: add Condition.wait_for(). Patch by sbt.
Charles-François Natali <neologix@free.fr>
parents: 74145
diff changeset
978 return result
5606ee052783 Issue #14087: multiprocessing: add Condition.wait_for(). Patch by sbt.
Charles-François Natali <neologix@free.fr>
parents: 74145
diff changeset
979 if timeout is not None:
5606ee052783 Issue #14087: multiprocessing: add Condition.wait_for(). Patch by sbt.
Charles-François Natali <neologix@free.fr>
parents: 74145
diff changeset
980 endtime = _time() + timeout
5606ee052783 Issue #14087: multiprocessing: add Condition.wait_for(). Patch by sbt.
Charles-François Natali <neologix@free.fr>
parents: 74145
diff changeset
981 else:
5606ee052783 Issue #14087: multiprocessing: add Condition.wait_for(). Patch by sbt.
Charles-François Natali <neologix@free.fr>
parents: 74145
diff changeset
982 endtime = None
5606ee052783 Issue #14087: multiprocessing: add Condition.wait_for(). Patch by sbt.
Charles-François Natali <neologix@free.fr>
parents: 74145
diff changeset
983 waittime = None
5606ee052783 Issue #14087: multiprocessing: add Condition.wait_for(). Patch by sbt.
Charles-François Natali <neologix@free.fr>
parents: 74145
diff changeset
984 while not result:
5606ee052783 Issue #14087: multiprocessing: add Condition.wait_for(). Patch by sbt.
Charles-François Natali <neologix@free.fr>
parents: 74145
diff changeset
985 if endtime is not None:
5606ee052783 Issue #14087: multiprocessing: add Condition.wait_for(). Patch by sbt.
Charles-François Natali <neologix@free.fr>
parents: 74145
diff changeset
986 waittime = endtime - _time()
5606ee052783 Issue #14087: multiprocessing: add Condition.wait_for(). Patch by sbt.
Charles-François Natali <neologix@free.fr>
parents: 74145
diff changeset
987 if waittime <= 0:
5606ee052783 Issue #14087: multiprocessing: add Condition.wait_for(). Patch by sbt.
Charles-François Natali <neologix@free.fr>
parents: 74145
diff changeset
988 break
5606ee052783 Issue #14087: multiprocessing: add Condition.wait_for(). Patch by sbt.
Charles-François Natali <neologix@free.fr>
parents: 74145
diff changeset
989 self.wait(waittime)
5606ee052783 Issue #14087: multiprocessing: add Condition.wait_for(). Patch by sbt.
Charles-François Natali <neologix@free.fr>
parents: 74145
diff changeset
990 result = predicate()
5606ee052783 Issue #14087: multiprocessing: add Condition.wait_for(). Patch by sbt.
Charles-François Natali <neologix@free.fr>
parents: 74145
diff changeset
991 return result
5606ee052783 Issue #14087: multiprocessing: add Condition.wait_for(). Patch by sbt.
Charles-François Natali <neologix@free.fr>
parents: 74145
diff changeset
992
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
993
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
994 class EventProxy(BaseProxy):
47642
5a9cff7fd08f attempt to fix multiprocessing
Benjamin Peterson <benjamin@python.org>
parents: 47615
diff changeset
995 _exposed_ = ('is_set', 'set', 'clear', 'wait')
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
996 def is_set(self):
47642
5a9cff7fd08f attempt to fix multiprocessing
Benjamin Peterson <benjamin@python.org>
parents: 47615
diff changeset
997 return self._callmethod('is_set')
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
998 def set(self):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
999 return self._callmethod('set')
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1000 def clear(self):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1001 return self._callmethod('clear')
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1002 def wait(self, timeout=None):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1003 return self._callmethod('wait', (timeout,))
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1004
77442
2d2f206d040e Issue #14059: Implement multiprocessing.Barrier
Richard Oudkerk <shibturn@gmail.com>
parents: 77428
diff changeset
1005
2d2f206d040e Issue #14059: Implement multiprocessing.Barrier
Richard Oudkerk <shibturn@gmail.com>
parents: 77428
diff changeset
1006 class BarrierProxy(BaseProxy):
2d2f206d040e Issue #14059: Implement multiprocessing.Barrier
Richard Oudkerk <shibturn@gmail.com>
parents: 77428
diff changeset
1007 _exposed_ = ('__getattribute__', 'wait', 'abort', 'reset')
2d2f206d040e Issue #14059: Implement multiprocessing.Barrier
Richard Oudkerk <shibturn@gmail.com>
parents: 77428
diff changeset
1008 def wait(self, timeout=None):
2d2f206d040e Issue #14059: Implement multiprocessing.Barrier
Richard Oudkerk <shibturn@gmail.com>
parents: 77428
diff changeset
1009 return self._callmethod('wait', (timeout,))
2d2f206d040e Issue #14059: Implement multiprocessing.Barrier
Richard Oudkerk <shibturn@gmail.com>
parents: 77428
diff changeset
1010 def abort(self):
2d2f206d040e Issue #14059: Implement multiprocessing.Barrier
Richard Oudkerk <shibturn@gmail.com>
parents: 77428
diff changeset
1011 return self._callmethod('abort')
2d2f206d040e Issue #14059: Implement multiprocessing.Barrier
Richard Oudkerk <shibturn@gmail.com>
parents: 77428
diff changeset
1012 def reset(self):
2d2f206d040e Issue #14059: Implement multiprocessing.Barrier
Richard Oudkerk <shibturn@gmail.com>
parents: 77428
diff changeset
1013 return self._callmethod('reset')
2d2f206d040e Issue #14059: Implement multiprocessing.Barrier
Richard Oudkerk <shibturn@gmail.com>
parents: 77428
diff changeset
1014 @property
2d2f206d040e Issue #14059: Implement multiprocessing.Barrier
Richard Oudkerk <shibturn@gmail.com>
parents: 77428
diff changeset
1015 def parties(self):
2d2f206d040e Issue #14059: Implement multiprocessing.Barrier
Richard Oudkerk <shibturn@gmail.com>
parents: 77428
diff changeset
1016 return self._callmethod('__getattribute__', ('parties',))
2d2f206d040e Issue #14059: Implement multiprocessing.Barrier
Richard Oudkerk <shibturn@gmail.com>
parents: 77428
diff changeset
1017 @property
2d2f206d040e Issue #14059: Implement multiprocessing.Barrier
Richard Oudkerk <shibturn@gmail.com>
parents: 77428
diff changeset
1018 def n_waiting(self):
2d2f206d040e Issue #14059: Implement multiprocessing.Barrier
Richard Oudkerk <shibturn@gmail.com>
parents: 77428
diff changeset
1019 return self._callmethod('__getattribute__', ('n_waiting',))
2d2f206d040e Issue #14059: Implement multiprocessing.Barrier
Richard Oudkerk <shibturn@gmail.com>
parents: 77428
diff changeset
1020 @property
2d2f206d040e Issue #14059: Implement multiprocessing.Barrier
Richard Oudkerk <shibturn@gmail.com>
parents: 77428
diff changeset
1021 def broken(self):
2d2f206d040e Issue #14059: Implement multiprocessing.Barrier
Richard Oudkerk <shibturn@gmail.com>
parents: 77428
diff changeset
1022 return self._callmethod('__getattribute__', ('broken',))
2d2f206d040e Issue #14059: Implement multiprocessing.Barrier
Richard Oudkerk <shibturn@gmail.com>
parents: 77428
diff changeset
1023
2d2f206d040e Issue #14059: Implement multiprocessing.Barrier
Richard Oudkerk <shibturn@gmail.com>
parents: 77428
diff changeset
1024
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1025 class NamespaceProxy(BaseProxy):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1026 _exposed_ = ('__getattribute__', '__setattr__', '__delattr__')
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1027 def __getattr__(self, key):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1028 if key[0] == '_':
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1029 return object.__getattribute__(self, key)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1030 callmethod = object.__getattribute__(self, '_callmethod')
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1031 return callmethod('__getattribute__', (key,))
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1032 def __setattr__(self, key, value):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1033 if key[0] == '_':
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1034 return object.__setattr__(self, key, value)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1035 callmethod = object.__getattribute__(self, '_callmethod')
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1036 return callmethod('__setattr__', (key, value))
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1037 def __delattr__(self, key):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1038 if key[0] == '_':
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1039 return object.__delattr__(self, key)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1040 callmethod = object.__getattribute__(self, '_callmethod')
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1041 return callmethod('__delattr__', (key,))
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1042
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1043
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1044 class ValueProxy(BaseProxy):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1045 _exposed_ = ('get', 'set')
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1046 def get(self):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1047 return self._callmethod('get')
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1048 def set(self, value):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1049 return self._callmethod('set', (value,))
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1050 value = property(get, set)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1051
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1052
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1053 BaseListProxy = MakeProxyType('BaseListProxy', (
77220
1f7c452e797e Remove __getslice__, __setslice__, __delslice__ methods from proxies
Richard Oudkerk <shibturn@gmail.com>
parents: 76800
diff changeset
1054 '__add__', '__contains__', '__delitem__', '__getitem__', '__len__',
1f7c452e797e Remove __getslice__, __setslice__, __delslice__ methods from proxies
Richard Oudkerk <shibturn@gmail.com>
parents: 76800
diff changeset
1055 '__mul__', '__reversed__', '__rmul__', '__setitem__',
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1056 'append', 'count', 'extend', 'index', 'insert', 'pop', 'remove',
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1057 'reverse', 'sort', '__imul__'
77220
1f7c452e797e Remove __getslice__, __setslice__, __delslice__ methods from proxies
Richard Oudkerk <shibturn@gmail.com>
parents: 76800
diff changeset
1058 ))
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1059 class ListProxy(BaseListProxy):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1060 def __iadd__(self, value):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1061 self._callmethod('extend', (value,))
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1062 return self
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1063 def __imul__(self, value):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1064 self._callmethod('__imul__', (value,))
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1065 return self
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1066
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1067
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1068 DictProxy = MakeProxyType('DictProxy', (
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1069 '__contains__', '__delitem__', '__getitem__', '__len__',
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1070 '__setitem__', 'clear', 'copy', 'get', 'has_key', 'items',
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1071 'keys', 'pop', 'popitem', 'setdefault', 'update', 'values'
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1072 ))
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1073
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1074
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1075 ArrayProxy = MakeProxyType('ArrayProxy', (
77220
1f7c452e797e Remove __getslice__, __setslice__, __delslice__ methods from proxies
Richard Oudkerk <shibturn@gmail.com>
parents: 76800
diff changeset
1076 '__len__', '__getitem__', '__setitem__'
1f7c452e797e Remove __getslice__, __setslice__, __delslice__ methods from proxies
Richard Oudkerk <shibturn@gmail.com>
parents: 76800
diff changeset
1077 ))
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1078
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1079
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1080 PoolProxy = MakeProxyType('PoolProxy', (
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1081 'apply', 'apply_async', 'close', 'imap', 'imap_unordered', 'join',
74107
b07b1e58582d Issue #12708: Add starmap() and starmap_async() methods (similar to itertools.starmap()) to multiprocessing.Pool.
Antoine Pitrou <solipsis@pitrou.net>
parents: 73513
diff changeset
1082 'map', 'map_async', 'starmap', 'starmap_async', 'terminate'
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1083 ))
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1084 PoolProxy._method_to_typeid_ = {
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1085 'apply_async': 'AsyncResult',
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1086 'map_async': 'AsyncResult',
74107
b07b1e58582d Issue #12708: Add starmap() and starmap_async() methods (similar to itertools.starmap()) to multiprocessing.Pool.
Antoine Pitrou <solipsis@pitrou.net>
parents: 73513
diff changeset
1087 'starmap_async': 'AsyncResult',
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1088 'imap': 'Iterator',
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1089 'imap_unordered': 'Iterator'
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1090 }
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1091
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1092 #
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1093 # Definition of SyncManager
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1094 #
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1095
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1096 class SyncManager(BaseManager):
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1097 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1098 Subclass of `BaseManager` which supports a number of shared object types.
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1099
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1100 The types registered are those intended for the synchronization
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1101 of threads, plus `dict`, `list` and `Namespace`.
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1102
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1103 The `multiprocessing.Manager()` function creates started instances of
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1104 this class.
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1105 '''
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1106
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1107 SyncManager.register('Queue', queue.Queue)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1108 SyncManager.register('JoinableQueue', queue.Queue)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1109 SyncManager.register('Event', threading.Event, EventProxy)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1110 SyncManager.register('Lock', threading.Lock, AcquirerProxy)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1111 SyncManager.register('RLock', threading.RLock, AcquirerProxy)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1112 SyncManager.register('Semaphore', threading.Semaphore, AcquirerProxy)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1113 SyncManager.register('BoundedSemaphore', threading.BoundedSemaphore,
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1114 AcquirerProxy)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1115 SyncManager.register('Condition', threading.Condition, ConditionProxy)
77442
2d2f206d040e Issue #14059: Implement multiprocessing.Barrier
Richard Oudkerk <shibturn@gmail.com>
parents: 77428
diff changeset
1116 SyncManager.register('Barrier', threading.Barrier, BarrierProxy)
85169
3b82e0d83bf9 Issue #8713: Support alternative start methods in multiprocessing on Unix.
Richard Oudkerk <shibturn@gmail.com>
parents: 84430
diff changeset
1117 SyncManager.register('Pool', pool.Pool, PoolProxy)
47595
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1118 SyncManager.register('list', list, ListProxy)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1119 SyncManager.register('dict', dict, DictProxy)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1120 SyncManager.register('Value', Value, ValueProxy)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1121 SyncManager.register('Array', Array, ArrayProxy)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1122 SyncManager.register('Namespace', Namespace, NamespaceProxy)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1123
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1124 # types returned by methods of PoolProxy
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1125 SyncManager.register('Iterator', proxytype=IteratorProxy, create_method=False)
5c6cdfcd6eae Merged revisions 64104,64117 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
diff changeset
1126 SyncManager.register('AsyncResult', create_method=False)