cpython-lru_cache-weakref: Lib/Queue.py annotate

annotate Lib/Queue.py @ 32527:0da660e05e69 legacy-trunk

"#if WITH_THREAD" is incorrect; must be #ifdef instead; WITH_THREAD isn't always set to an integer value when it's defined.

author	Tim Peters <tim.peters@gmail.com>
date	Wed, 07 Jul 2004 20:42:07 +0000
parents	63fa60f2d61c
children	73016bd01a71

rev	line source
11691 e34991022571 Mass patch by Ka-Ping Yee: Guido van Rossum <guido@python.org> parents: 11267 diff changeset	1 """A multi-producer, multi-consumer queue."""
995 f025eef93130 New module implementing a multi-everything queue. Guido van Rossum <guido@python.org> parents: diff changeset	2
25796 3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	3 from time import time as _time, sleep as _sleep
31436 63fa60f2d61c * Move collections.deque() in from the sandbox Raymond Hettinger <python@rcn.com> parents: 29711 diff changeset	4 from collections import deque
25796 3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	5
29711 58132c813d7b Add __all__ . Brett Cannon <bcannon@gmail.com> parents: 26892 diff changeset	6 __all__ = ['Empty', 'Full', 'Queue']
58132c813d7b Add __all__ . Brett Cannon <bcannon@gmail.com> parents: 26892 diff changeset	7
15902 eef321b6e41c Variant of Skip's patch 103246 (Remove unneeded string exception compat from Queue). Tim Peters <tim.peters@gmail.com> parents: 13050 diff changeset	8 class Empty(Exception):
eef321b6e41c Variant of Skip's patch 103246 (Remove unneeded string exception compat from Queue). Tim Peters <tim.peters@gmail.com> parents: 13050 diff changeset	9 "Exception raised by Queue.get(block=0)/get_nowait()."
eef321b6e41c Variant of Skip's patch 103246 (Remove unneeded string exception compat from Queue). Tim Peters <tim.peters@gmail.com> parents: 13050 diff changeset	10 pass
eef321b6e41c Variant of Skip's patch 103246 (Remove unneeded string exception compat from Queue). Tim Peters <tim.peters@gmail.com> parents: 13050 diff changeset	11
eef321b6e41c Variant of Skip's patch 103246 (Remove unneeded string exception compat from Queue). Tim Peters <tim.peters@gmail.com> parents: 13050 diff changeset	12 class Full(Exception):
eef321b6e41c Variant of Skip's patch 103246 (Remove unneeded string exception compat from Queue). Tim Peters <tim.peters@gmail.com> parents: 13050 diff changeset	13 "Exception raised by Queue.put(block=0)/put_nowait()."
eef321b6e41c Variant of Skip's patch 103246 (Remove unneeded string exception compat from Queue). Tim Peters <tim.peters@gmail.com> parents: 13050 diff changeset	14 pass
995 f025eef93130 New module implementing a multi-everything queue. Guido van Rossum <guido@python.org> parents: diff changeset	15
f025eef93130 New module implementing a multi-everything queue. Guido van Rossum <guido@python.org> parents: diff changeset	16 class Queue:
11267 2b26e48a85a6 Make the maxsize constructor argument default to 0 (an unlimited queue size). Guido van Rossum <guido@python.org> parents: 10120 diff changeset	17 def __init__(self, maxsize=0):
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 6612 diff changeset	18 """Initialize a queue object with a given maximum size.
995 f025eef93130 New module implementing a multi-everything queue. Guido van Rossum <guido@python.org> parents: diff changeset	19
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 6612 diff changeset	20 If maxsize is <= 0, the queue size is infinite.
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 6612 diff changeset	21 """
26892 95510f0a0624 Use the dummy_thread module in Queue.py and tempfile.py. Guido van Rossum <guido@python.org> parents: 25796 diff changeset	22 try:
95510f0a0624 Use the dummy_thread module in Queue.py and tempfile.py. Guido van Rossum <guido@python.org> parents: 25796 diff changeset	23 import thread
95510f0a0624 Use the dummy_thread module in Queue.py and tempfile.py. Guido van Rossum <guido@python.org> parents: 25796 diff changeset	24 except ImportError:
95510f0a0624 Use the dummy_thread module in Queue.py and tempfile.py. Guido van Rossum <guido@python.org> parents: 25796 diff changeset	25 import dummy_thread as thread
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 6612 diff changeset	26 self._init(maxsize)
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 6612 diff changeset	27 self.mutex = thread.allocate_lock()
fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 6612 diff changeset	28 self.esema = thread.allocate_lock()
8197 3a55b4c5e3c7 Replace all calls to acquire_lock() and release_lock() with acquire() Guido van Rossum <guido@python.org> parents: 7926 diff changeset	29 self.esema.acquire()
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 6612 diff changeset	30 self.fsema = thread.allocate_lock()
995 f025eef93130 New module implementing a multi-everything queue. Guido van Rossum <guido@python.org> parents: diff changeset	31
6612 4b2d8dc92db4 (Queue.Empty): When class based exceptions are in force, derive this Barry Warsaw <barry@python.org> parents: 1463 diff changeset	32 def qsize(self):
10120 5b15642e7bdf Tim Peters: Guido van Rossum <guido@python.org> parents: 8197 diff changeset	33 """Return the approximate size of the queue (not reliable!)."""
8197 3a55b4c5e3c7 Replace all calls to acquire_lock() and release_lock() with acquire() Guido van Rossum <guido@python.org> parents: 7926 diff changeset	34 self.mutex.acquire()
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 6612 diff changeset	35 n = self._qsize()
8197 3a55b4c5e3c7 Replace all calls to acquire_lock() and release_lock() with acquire() Guido van Rossum <guido@python.org> parents: 7926 diff changeset	36 self.mutex.release()
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 6612 diff changeset	37 return n
995 f025eef93130 New module implementing a multi-everything queue. Guido van Rossum <guido@python.org> parents: diff changeset	38
6612 4b2d8dc92db4 (Queue.Empty): When class based exceptions are in force, derive this Barry Warsaw <barry@python.org> parents: 1463 diff changeset	39 def empty(self):
25796 3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	40 """Return True if the queue is empty, False otherwise (not reliable!)."""
8197 3a55b4c5e3c7 Replace all calls to acquire_lock() and release_lock() with acquire() Guido van Rossum <guido@python.org> parents: 7926 diff changeset	41 self.mutex.acquire()
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 6612 diff changeset	42 n = self._empty()
8197 3a55b4c5e3c7 Replace all calls to acquire_lock() and release_lock() with acquire() Guido van Rossum <guido@python.org> parents: 7926 diff changeset	43 self.mutex.release()
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 6612 diff changeset	44 return n
995 f025eef93130 New module implementing a multi-everything queue. Guido van Rossum <guido@python.org> parents: diff changeset	45
6612 4b2d8dc92db4 (Queue.Empty): When class based exceptions are in force, derive this Barry Warsaw <barry@python.org> parents: 1463 diff changeset	46 def full(self):
25796 3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	47 """Return True if the queue is full, False otherwise (not reliable!)."""
8197 3a55b4c5e3c7 Replace all calls to acquire_lock() and release_lock() with acquire() Guido van Rossum <guido@python.org> parents: 7926 diff changeset	48 self.mutex.acquire()
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 6612 diff changeset	49 n = self._full()
8197 3a55b4c5e3c7 Replace all calls to acquire_lock() and release_lock() with acquire() Guido van Rossum <guido@python.org> parents: 7926 diff changeset	50 self.mutex.release()
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 6612 diff changeset	51 return n
995 f025eef93130 New module implementing a multi-everything queue. Guido van Rossum <guido@python.org> parents: diff changeset	52
25796 3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	53 def put(self, item, block=True, timeout=None):
7926 e29d435ab142 Clarify that put blocks when the queue is full. Add some blank Guido van Rossum <guido@python.org> parents: 7755 diff changeset	54 """Put an item into the queue.
e29d435ab142 Clarify that put blocks when the queue is full. Add some blank Guido van Rossum <guido@python.org> parents: 7755 diff changeset	55
25796 3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	56 If optional args 'block' is true and 'timeout' is None (the default),
3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	57 block if necessary until a free slot is available. If 'timeout' is
3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	58 a positive number, it blocks at most 'timeout' seconds and raises
3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	59 the Full exception if no free slot was available within that time.
3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	60 Otherwise ('block' is false), put an item on the queue if a free slot
3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	61 is immediately available, else raise the Full exception ('timeout'
3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	62 is ignored in that case).
10120 5b15642e7bdf Tim Peters: Guido van Rossum <guido@python.org> parents: 8197 diff changeset	63 """
5b15642e7bdf Tim Peters: Guido van Rossum <guido@python.org> parents: 8197 diff changeset	64 if block:
25796 3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	65 if timeout is None:
3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	66 # blocking, w/o timeout, i.e. forever
3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	67 self.fsema.acquire()
3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	68 elif timeout >= 0:
3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	69 # waiting max. 'timeout' seconds.
3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	70 # this code snipped is from threading.py: _Event.wait():
3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	71 # Balancing act: We can't afford a pure busy loop, so we
3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	72 # have to sleep; but if we sleep the whole timeout time,
3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	73 # we'll be unresponsive. The scheme here sleeps very
3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	74 # little at first, longer as time goes on, but never longer
3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	75 # than 20 times per second (or the timeout time remaining).
3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	76 delay = 0.0005 # 500 us -> initial delay of 1 ms
3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	77 endtime = _time() + timeout
3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	78 while True:
3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	79 if self.fsema.acquire(0):
3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	80 break
3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	81 remaining = endtime - _time()
3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	82 if remaining <= 0: #time is over and no slot was free
3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	83 raise Full
3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	84 delay = min(delay * 2, remaining, .05)
3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	85 _sleep(delay) #reduce CPU usage by using a sleep
3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	86 else:
3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	87 raise ValueError("'timeout' must be a positive number")
10120 5b15642e7bdf Tim Peters: Guido van Rossum <guido@python.org> parents: 8197 diff changeset	88 elif not self.fsema.acquire(0):
5b15642e7bdf Tim Peters: Guido van Rossum <guido@python.org> parents: 8197 diff changeset	89 raise Full
8197 3a55b4c5e3c7 Replace all calls to acquire_lock() and release_lock() with acquire() Guido van Rossum <guido@python.org> parents: 7926 diff changeset	90 self.mutex.acquire()
23160 c548e726b3bc Fix bug 544473 - "Queue module can deadlock". Mark Hammond <mhammond@skippinet.com.au> parents: 16657 diff changeset	91 release_fsema = True
c548e726b3bc Fix bug 544473 - "Queue module can deadlock". Mark Hammond <mhammond@skippinet.com.au> parents: 16657 diff changeset	92 try:
c548e726b3bc Fix bug 544473 - "Queue module can deadlock". Mark Hammond <mhammond@skippinet.com.au> parents: 16657 diff changeset	93 was_empty = self._empty()
c548e726b3bc Fix bug 544473 - "Queue module can deadlock". Mark Hammond <mhammond@skippinet.com.au> parents: 16657 diff changeset	94 self._put(item)
c548e726b3bc Fix bug 544473 - "Queue module can deadlock". Mark Hammond <mhammond@skippinet.com.au> parents: 16657 diff changeset	95 # If we fail before here, the empty state has
c548e726b3bc Fix bug 544473 - "Queue module can deadlock". Mark Hammond <mhammond@skippinet.com.au> parents: 16657 diff changeset	96 # not changed, so we can skip the release of esema
c548e726b3bc Fix bug 544473 - "Queue module can deadlock". Mark Hammond <mhammond@skippinet.com.au> parents: 16657 diff changeset	97 if was_empty:
c548e726b3bc Fix bug 544473 - "Queue module can deadlock". Mark Hammond <mhammond@skippinet.com.au> parents: 16657 diff changeset	98 self.esema.release()
c548e726b3bc Fix bug 544473 - "Queue module can deadlock". Mark Hammond <mhammond@skippinet.com.au> parents: 16657 diff changeset	99 # If we fail before here, the queue can not be full, so
c548e726b3bc Fix bug 544473 - "Queue module can deadlock". Mark Hammond <mhammond@skippinet.com.au> parents: 16657 diff changeset	100 # release_full_sema remains True
c548e726b3bc Fix bug 544473 - "Queue module can deadlock". Mark Hammond <mhammond@skippinet.com.au> parents: 16657 diff changeset	101 release_fsema = not self._full()
c548e726b3bc Fix bug 544473 - "Queue module can deadlock". Mark Hammond <mhammond@skippinet.com.au> parents: 16657 diff changeset	102 finally:
c548e726b3bc Fix bug 544473 - "Queue module can deadlock". Mark Hammond <mhammond@skippinet.com.au> parents: 16657 diff changeset	103 # Catching system level exceptions here (RecursionDepth,
c548e726b3bc Fix bug 544473 - "Queue module can deadlock". Mark Hammond <mhammond@skippinet.com.au> parents: 16657 diff changeset	104 # OutOfMemory, etc) - so do as little as possible in terms
c548e726b3bc Fix bug 544473 - "Queue module can deadlock". Mark Hammond <mhammond@skippinet.com.au> parents: 16657 diff changeset	105 # of Python calls.
c548e726b3bc Fix bug 544473 - "Queue module can deadlock". Mark Hammond <mhammond@skippinet.com.au> parents: 16657 diff changeset	106 if release_fsema:
c548e726b3bc Fix bug 544473 - "Queue module can deadlock". Mark Hammond <mhammond@skippinet.com.au> parents: 16657 diff changeset	107 self.fsema.release()
c548e726b3bc Fix bug 544473 - "Queue module can deadlock". Mark Hammond <mhammond@skippinet.com.au> parents: 16657 diff changeset	108 self.mutex.release()
995 f025eef93130 New module implementing a multi-everything queue. Guido van Rossum <guido@python.org> parents: diff changeset	109
10120 5b15642e7bdf Tim Peters: Guido van Rossum <guido@python.org> parents: 8197 diff changeset	110 def put_nowait(self, item):
5b15642e7bdf Tim Peters: Guido van Rossum <guido@python.org> parents: 8197 diff changeset	111 """Put an item into the queue without blocking.
5b15642e7bdf Tim Peters: Guido van Rossum <guido@python.org> parents: 8197 diff changeset	112
5b15642e7bdf Tim Peters: Guido van Rossum <guido@python.org> parents: 8197 diff changeset	113 Only enqueue the item if a free slot is immediately available.
5b15642e7bdf Tim Peters: Guido van Rossum <guido@python.org> parents: 8197 diff changeset	114 Otherwise raise the Full exception.
5b15642e7bdf Tim Peters: Guido van Rossum <guido@python.org> parents: 8197 diff changeset	115 """
25796 3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	116 return self.put(item, False)
7926 e29d435ab142 Clarify that put blocks when the queue is full. Add some blank Guido van Rossum <guido@python.org> parents: 7755 diff changeset	117
25796 3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	118 def get(self, block=True, timeout=None):
10120 5b15642e7bdf Tim Peters: Guido van Rossum <guido@python.org> parents: 8197 diff changeset	119 """Remove and return an item from the queue.
5b15642e7bdf Tim Peters: Guido van Rossum <guido@python.org> parents: 8197 diff changeset	120
25796 3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	121 If optional args 'block' is true and 'timeout' is None (the default),
3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	122 block if necessary until an item is available. If 'timeout' is
3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	123 a positive number, it blocks at most 'timeout' seconds and raises
3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	124 the Empty exception if no item was available within that time.
3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	125 Otherwise ('block' is false), return an item if one is immediately
3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	126 available, else raise the Empty exception ('timeout' is ignored
3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	127 in that case).
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 6612 diff changeset	128 """
10120 5b15642e7bdf Tim Peters: Guido van Rossum <guido@python.org> parents: 8197 diff changeset	129 if block:
25796 3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	130 if timeout is None:
3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	131 # blocking, w/o timeout, i.e. forever
3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	132 self.esema.acquire()
3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	133 elif timeout >= 0:
3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	134 # waiting max. 'timeout' seconds.
3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	135 # this code snipped is from threading.py: _Event.wait():
3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	136 # Balancing act: We can't afford a pure busy loop, so we
3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	137 # have to sleep; but if we sleep the whole timeout time,
3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	138 # we'll be unresponsive. The scheme here sleeps very
3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	139 # little at first, longer as time goes on, but never longer
3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	140 # than 20 times per second (or the timeout time remaining).
3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	141 delay = 0.0005 # 500 us -> initial delay of 1 ms
3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	142 endtime = _time() + timeout
3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	143 while 1:
3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	144 if self.esema.acquire(0):
3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	145 break
3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	146 remaining = endtime - _time()
3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	147 if remaining <= 0: #time is over and no element arrived
3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	148 raise Empty
3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	149 delay = min(delay * 2, remaining, .05)
3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	150 _sleep(delay) #reduce CPU usage by using a sleep
3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	151 else:
3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	152 raise ValueError("'timeout' must be a positive number")
10120 5b15642e7bdf Tim Peters: Guido van Rossum <guido@python.org> parents: 8197 diff changeset	153 elif not self.esema.acquire(0):
5b15642e7bdf Tim Peters: Guido van Rossum <guido@python.org> parents: 8197 diff changeset	154 raise Empty
8197 3a55b4c5e3c7 Replace all calls to acquire_lock() and release_lock() with acquire() Guido van Rossum <guido@python.org> parents: 7926 diff changeset	155 self.mutex.acquire()
23160 c548e726b3bc Fix bug 544473 - "Queue module can deadlock". Mark Hammond <mhammond@skippinet.com.au> parents: 16657 diff changeset	156 release_esema = True
c548e726b3bc Fix bug 544473 - "Queue module can deadlock". Mark Hammond <mhammond@skippinet.com.au> parents: 16657 diff changeset	157 try:
c548e726b3bc Fix bug 544473 - "Queue module can deadlock". Mark Hammond <mhammond@skippinet.com.au> parents: 16657 diff changeset	158 was_full = self._full()
c548e726b3bc Fix bug 544473 - "Queue module can deadlock". Mark Hammond <mhammond@skippinet.com.au> parents: 16657 diff changeset	159 item = self._get()
c548e726b3bc Fix bug 544473 - "Queue module can deadlock". Mark Hammond <mhammond@skippinet.com.au> parents: 16657 diff changeset	160 # If we fail before here, the full state has
c548e726b3bc Fix bug 544473 - "Queue module can deadlock". Mark Hammond <mhammond@skippinet.com.au> parents: 16657 diff changeset	161 # not changed, so we can skip the release of fsema
c548e726b3bc Fix bug 544473 - "Queue module can deadlock". Mark Hammond <mhammond@skippinet.com.au> parents: 16657 diff changeset	162 if was_full:
c548e726b3bc Fix bug 544473 - "Queue module can deadlock". Mark Hammond <mhammond@skippinet.com.au> parents: 16657 diff changeset	163 self.fsema.release()
c548e726b3bc Fix bug 544473 - "Queue module can deadlock". Mark Hammond <mhammond@skippinet.com.au> parents: 16657 diff changeset	164 # Failure means empty state also unchanged - release_esema
c548e726b3bc Fix bug 544473 - "Queue module can deadlock". Mark Hammond <mhammond@skippinet.com.au> parents: 16657 diff changeset	165 # remains True.
c548e726b3bc Fix bug 544473 - "Queue module can deadlock". Mark Hammond <mhammond@skippinet.com.au> parents: 16657 diff changeset	166 release_esema = not self._empty()
c548e726b3bc Fix bug 544473 - "Queue module can deadlock". Mark Hammond <mhammond@skippinet.com.au> parents: 16657 diff changeset	167 finally:
c548e726b3bc Fix bug 544473 - "Queue module can deadlock". Mark Hammond <mhammond@skippinet.com.au> parents: 16657 diff changeset	168 if release_esema:
c548e726b3bc Fix bug 544473 - "Queue module can deadlock". Mark Hammond <mhammond@skippinet.com.au> parents: 16657 diff changeset	169 self.esema.release()
c548e726b3bc Fix bug 544473 - "Queue module can deadlock". Mark Hammond <mhammond@skippinet.com.au> parents: 16657 diff changeset	170 self.mutex.release()
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 6612 diff changeset	171 return item
995 f025eef93130 New module implementing a multi-everything queue. Guido van Rossum <guido@python.org> parents: diff changeset	172
6612 4b2d8dc92db4 (Queue.Empty): When class based exceptions are in force, derive this Barry Warsaw <barry@python.org> parents: 1463 diff changeset	173 def get_nowait(self):
10120 5b15642e7bdf Tim Peters: Guido van Rossum <guido@python.org> parents: 8197 diff changeset	174 """Remove and return an item from the queue without blocking.
7926 e29d435ab142 Clarify that put blocks when the queue is full. Add some blank Guido van Rossum <guido@python.org> parents: 7755 diff changeset	175
25796 3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	176 Only get an item if one is immediately available. Otherwise
10120 5b15642e7bdf Tim Peters: Guido van Rossum <guido@python.org> parents: 8197 diff changeset	177 raise the Empty exception.
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 6612 diff changeset	178 """
25796 3eafbde56b21 Patch #572628: Optional timeouts for put and get. Martin v. Löwis <martin@v.loewis.de> parents: 24058 diff changeset	179 return self.get(False)
6612 4b2d8dc92db4 (Queue.Empty): When class based exceptions are in force, derive this Barry Warsaw <barry@python.org> parents: 1463 diff changeset	180
4b2d8dc92db4 (Queue.Empty): When class based exceptions are in force, derive this Barry Warsaw <barry@python.org> parents: 1463 diff changeset	181 # Override these methods to implement other queue organizations
4b2d8dc92db4 (Queue.Empty): When class based exceptions are in force, derive this Barry Warsaw <barry@python.org> parents: 1463 diff changeset	182 # (e.g. stack or priority queue).
4b2d8dc92db4 (Queue.Empty): When class based exceptions are in force, derive this Barry Warsaw <barry@python.org> parents: 1463 diff changeset	183 # These will only be called with appropriate locks held
995 f025eef93130 New module implementing a multi-everything queue. Guido van Rossum <guido@python.org> parents: diff changeset	184
6612 4b2d8dc92db4 (Queue.Empty): When class based exceptions are in force, derive this Barry Warsaw <barry@python.org> parents: 1463 diff changeset	185 # Initialize the queue representation
4b2d8dc92db4 (Queue.Empty): When class based exceptions are in force, derive this Barry Warsaw <barry@python.org> parents: 1463 diff changeset	186 def _init(self, maxsize):
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 6612 diff changeset	187 self.maxsize = maxsize
31436 63fa60f2d61c * Move collections.deque() in from the sandbox Raymond Hettinger <python@rcn.com> parents: 29711 diff changeset	188 self.queue = deque()
995 f025eef93130 New module implementing a multi-everything queue. Guido van Rossum <guido@python.org> parents: diff changeset	189
6612 4b2d8dc92db4 (Queue.Empty): When class based exceptions are in force, derive this Barry Warsaw <barry@python.org> parents: 1463 diff changeset	190 def _qsize(self):
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 6612 diff changeset	191 return len(self.queue)
995 f025eef93130 New module implementing a multi-everything queue. Guido van Rossum <guido@python.org> parents: diff changeset	192
13050 a195962e389c typos fixed by Rob Hooft Jeremy Hylton <jeremy@alum.mit.edu> parents: 11691 diff changeset	193 # Check whether the queue is empty
6612 4b2d8dc92db4 (Queue.Empty): When class based exceptions are in force, derive this Barry Warsaw <barry@python.org> parents: 1463 diff changeset	194 def _empty(self):
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 6612 diff changeset	195 return not self.queue
995 f025eef93130 New module implementing a multi-everything queue. Guido van Rossum <guido@python.org> parents: diff changeset	196
6612 4b2d8dc92db4 (Queue.Empty): When class based exceptions are in force, derive this Barry Warsaw <barry@python.org> parents: 1463 diff changeset	197 # Check whether the queue is full
4b2d8dc92db4 (Queue.Empty): When class based exceptions are in force, derive this Barry Warsaw <barry@python.org> parents: 1463 diff changeset	198 def _full(self):
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 6612 diff changeset	199 return self.maxsize > 0 and len(self.queue) == self.maxsize
995 f025eef93130 New module implementing a multi-everything queue. Guido van Rossum <guido@python.org> parents: diff changeset	200
6612 4b2d8dc92db4 (Queue.Empty): When class based exceptions are in force, derive this Barry Warsaw <barry@python.org> parents: 1463 diff changeset	201 # Put a new item in the queue
4b2d8dc92db4 (Queue.Empty): When class based exceptions are in force, derive this Barry Warsaw <barry@python.org> parents: 1463 diff changeset	202 def _put(self, item):
7755 fb4b4d069cc7 Mass check-in after untabifying all files that need it. Guido van Rossum <guido@python.org> parents: 6612 diff changeset	203 self.queue.append(item)
995 f025eef93130 New module implementing a multi-everything queue. Guido van Rossum <guido@python.org> parents: diff changeset	204
6612 4b2d8dc92db4 (Queue.Empty): When class based exceptions are in force, derive this Barry Warsaw <barry@python.org> parents: 1463 diff changeset	205 # Get an item from the queue
4b2d8dc92db4 (Queue.Empty): When class based exceptions are in force, derive this Barry Warsaw <barry@python.org> parents: 1463 diff changeset	206 def _get(self):
31436 63fa60f2d61c * Move collections.deque() in from the sandbox Raymond Hettinger <python@rcn.com> parents: 29711 diff changeset	207 return self.queue.popleft()

Mercurial > cpython-lru_cache-weakref

annotate Lib/Queue.py @ 32527:0da660e05e69 legacy-trunk