cpython-lru_cache-weakref: Lib/queue.py annotate

annotate Lib/queue.py @ 87690:66c1c9f32567 default tip

Add use_weakref option to functools.lru_cache The use_weakref option makes lru_cache create weak references to the function arguments when creating the cache key, which prevents them from staying in the memory forever even after all other references to them are gone. This is a proof of concept for http://bugs.python.org/issue19859

author	Radomir Dopieralski <devel@sheep.art.pl>
date	Tue, 03 Dec 2013 16:02:48 +0100
parents	d9a9fe5e700d
children

rev	line source
74307 ca2a35140e6a Make the docstring style consistent. Raymond Hettinger <python@rcn.com> parents: 74306 diff changeset	1 '''A multi-producer, multi-consumer queue.'''
46899 bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	2
52259 e41f5a89f5c0 port the queue change r70405 Benjamin Peterson <benjamin@python.org> parents: 52045 diff changeset	3 try:
74305 190ad17f5a87 Minor code clean-ups and beautifications. Raymond Hettinger <python@rcn.com> parents: 74298 diff changeset	4 import threading
52259 e41f5a89f5c0 port the queue change r70405 Benjamin Peterson <benjamin@python.org> parents: 52045 diff changeset	5 except ImportError:
74309 32eae3c48631 Fix nasty typo Raymond Hettinger <python@rcn.com> parents: 74307 diff changeset	6 import dummy_threading as threading
46899 bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	7 from collections import deque
74305 190ad17f5a87 Minor code clean-ups and beautifications. Raymond Hettinger <python@rcn.com> parents: 74298 diff changeset	8 from heapq import heappush, heappop
76597 76d2e0761d18 Issue #14428, #14397: Implement the PEP 418 Victor Stinner <victor.stinner@gmail.com> parents: 75675 diff changeset	9 try:
76d2e0761d18 Issue #14428, #14397: Implement the PEP 418 Victor Stinner <victor.stinner@gmail.com> parents: 75675 diff changeset	10 from time import monotonic as time
76d2e0761d18 Issue #14428, #14397: Implement the PEP 418 Victor Stinner <victor.stinner@gmail.com> parents: 75675 diff changeset	11 except ImportError:
76d2e0761d18 Issue #14428, #14397: Implement the PEP 418 Victor Stinner <victor.stinner@gmail.com> parents: 75675 diff changeset	12 from time import time
46899 bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	13
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	14 __all__ = ['Empty', 'Full', 'Queue', 'PriorityQueue', 'LifoQueue']
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	15
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	16 class Empty(Exception):
74307 ca2a35140e6a Make the docstring style consistent. Raymond Hettinger <python@rcn.com> parents: 74306 diff changeset	17 'Exception raised by Queue.get(block=0)/get_nowait().'
46899 bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	18 pass
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	19
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	20 class Full(Exception):
74307 ca2a35140e6a Make the docstring style consistent. Raymond Hettinger <python@rcn.com> parents: 74306 diff changeset	21 'Exception raised by Queue.put(block=0)/put_nowait().'
46899 bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	22 pass
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	23
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	24 class Queue:
74307 ca2a35140e6a Make the docstring style consistent. Raymond Hettinger <python@rcn.com> parents: 74306 diff changeset	25 '''Create a queue object with a given maximum size.
46899 bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	26
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	27 If maxsize is <= 0, the queue size is infinite.
74307 ca2a35140e6a Make the docstring style consistent. Raymond Hettinger <python@rcn.com> parents: 74306 diff changeset	28 '''
ca2a35140e6a Make the docstring style consistent. Raymond Hettinger <python@rcn.com> parents: 74306 diff changeset	29
46899 bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	30 def __init__(self, maxsize=0):
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	31 self.maxsize = maxsize
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	32 self._init(maxsize)
74298 c68426b241ca Simplify the code using with-statements. Raymond Hettinger <python@rcn.com> parents: 65831 diff changeset	33
46899 bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	34 # mutex must be held whenever the queue is mutating. All methods
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	35 # that acquire mutex must release it before returning. mutex
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	36 # is shared between the three conditions, so acquiring and
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	37 # releasing the conditions also acquires and releases mutex.
74305 190ad17f5a87 Minor code clean-ups and beautifications. Raymond Hettinger <python@rcn.com> parents: 74298 diff changeset	38 self.mutex = threading.Lock()
74298 c68426b241ca Simplify the code using with-statements. Raymond Hettinger <python@rcn.com> parents: 65831 diff changeset	39
46899 bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	40 # Notify not_empty whenever an item is added to the queue; a
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	41 # thread waiting to get is notified then.
74305 190ad17f5a87 Minor code clean-ups and beautifications. Raymond Hettinger <python@rcn.com> parents: 74298 diff changeset	42 self.not_empty = threading.Condition(self.mutex)
74298 c68426b241ca Simplify the code using with-statements. Raymond Hettinger <python@rcn.com> parents: 65831 diff changeset	43
46899 bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	44 # Notify not_full whenever an item is removed from the queue;
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	45 # a thread waiting to put is notified then.
74305 190ad17f5a87 Minor code clean-ups and beautifications. Raymond Hettinger <python@rcn.com> parents: 74298 diff changeset	46 self.not_full = threading.Condition(self.mutex)
74298 c68426b241ca Simplify the code using with-statements. Raymond Hettinger <python@rcn.com> parents: 65831 diff changeset	47
46899 bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	48 # Notify all_tasks_done whenever the number of unfinished tasks
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	49 # drops to zero; thread waiting to join() is notified to resume
74305 190ad17f5a87 Minor code clean-ups and beautifications. Raymond Hettinger <python@rcn.com> parents: 74298 diff changeset	50 self.all_tasks_done = threading.Condition(self.mutex)
46899 bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	51 self.unfinished_tasks = 0
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	52
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	53 def task_done(self):
74307 ca2a35140e6a Make the docstring style consistent. Raymond Hettinger <python@rcn.com> parents: 74306 diff changeset	54 '''Indicate that a formerly enqueued task is complete.
46899 bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	55
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	56 Used by Queue consumer threads. For each get() used to fetch a task,
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	57 a subsequent call to task_done() tells the queue that the processing
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	58 on the task is complete.
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	59
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	60 If a join() is currently blocking, it will resume when all items
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	61 have been processed (meaning that a task_done() call was received
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	62 for every item that had been put() into the queue).
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	63
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	64 Raises a ValueError if called more times than there were items
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	65 placed in the queue.
74307 ca2a35140e6a Make the docstring style consistent. Raymond Hettinger <python@rcn.com> parents: 74306 diff changeset	66 '''
74298 c68426b241ca Simplify the code using with-statements. Raymond Hettinger <python@rcn.com> parents: 65831 diff changeset	67 with self.all_tasks_done:
46899 bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	68 unfinished = self.unfinished_tasks - 1
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	69 if unfinished <= 0:
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	70 if unfinished < 0:
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	71 raise ValueError('task_done() called too many times')
47615 045f83a1b864 Merged revisions 64125 via svnmerge from Benjamin Peterson <benjamin@python.org> parents: 46899 diff changeset	72 self.all_tasks_done.notify_all()
46899 bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	73 self.unfinished_tasks = unfinished
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	74
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	75 def join(self):
74307 ca2a35140e6a Make the docstring style consistent. Raymond Hettinger <python@rcn.com> parents: 74306 diff changeset	76 '''Blocks until all items in the Queue have been gotten and processed.
46899 bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	77
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	78 The count of unfinished tasks goes up whenever an item is added to the
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	79 queue. The count goes down whenever a consumer thread calls task_done()
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	80 to indicate the item was retrieved and all work on it is complete.
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	81
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	82 When the count of unfinished tasks drops to zero, join() unblocks.
74307 ca2a35140e6a Make the docstring style consistent. Raymond Hettinger <python@rcn.com> parents: 74306 diff changeset	83 '''
74298 c68426b241ca Simplify the code using with-statements. Raymond Hettinger <python@rcn.com> parents: 65831 diff changeset	84 with self.all_tasks_done:
46899 bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	85 while self.unfinished_tasks:
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	86 self.all_tasks_done.wait()
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	87
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	88 def qsize(self):
74307 ca2a35140e6a Make the docstring style consistent. Raymond Hettinger <python@rcn.com> parents: 74306 diff changeset	89 '''Return the approximate size of the queue (not reliable!).'''
74298 c68426b241ca Simplify the code using with-statements. Raymond Hettinger <python@rcn.com> parents: 65831 diff changeset	90 with self.mutex:
c68426b241ca Simplify the code using with-statements. Raymond Hettinger <python@rcn.com> parents: 65831 diff changeset	91 return self._qsize()
46899 bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	92
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	93 def empty(self):
74307 ca2a35140e6a Make the docstring style consistent. Raymond Hettinger <python@rcn.com> parents: 74306 diff changeset	94 '''Return True if the queue is empty, False otherwise (not reliable!).
52045 aef5cfe73358 Document the suggested alternative to emtpy() and full(). Raymond Hettinger <python@rcn.com> parents: 47615 diff changeset	95
aef5cfe73358 Document the suggested alternative to emtpy() and full(). Raymond Hettinger <python@rcn.com> parents: 47615 diff changeset	96 This method is likely to be removed at some point. Use qsize() == 0
aef5cfe73358 Document the suggested alternative to emtpy() and full(). Raymond Hettinger <python@rcn.com> parents: 47615 diff changeset	97 as a direct substitute, but be aware that either approach risks a race
aef5cfe73358 Document the suggested alternative to emtpy() and full(). Raymond Hettinger <python@rcn.com> parents: 47615 diff changeset	98 condition where a queue can grow before the result of empty() or
aef5cfe73358 Document the suggested alternative to emtpy() and full(). Raymond Hettinger <python@rcn.com> parents: 47615 diff changeset	99 qsize() can be used.
aef5cfe73358 Document the suggested alternative to emtpy() and full(). Raymond Hettinger <python@rcn.com> parents: 47615 diff changeset	100
aef5cfe73358 Document the suggested alternative to emtpy() and full(). Raymond Hettinger <python@rcn.com> parents: 47615 diff changeset	101 To create code that needs to wait for all queued tasks to be
aef5cfe73358 Document the suggested alternative to emtpy() and full(). Raymond Hettinger <python@rcn.com> parents: 47615 diff changeset	102 completed, the preferred technique is to use the join() method.
74307 ca2a35140e6a Make the docstring style consistent. Raymond Hettinger <python@rcn.com> parents: 74306 diff changeset	103 '''
74298 c68426b241ca Simplify the code using with-statements. Raymond Hettinger <python@rcn.com> parents: 65831 diff changeset	104 with self.mutex:
c68426b241ca Simplify the code using with-statements. Raymond Hettinger <python@rcn.com> parents: 65831 diff changeset	105 return not self._qsize()
46899 bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	106
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	107 def full(self):
74307 ca2a35140e6a Make the docstring style consistent. Raymond Hettinger <python@rcn.com> parents: 74306 diff changeset	108 '''Return True if the queue is full, False otherwise (not reliable!).
52045 aef5cfe73358 Document the suggested alternative to emtpy() and full(). Raymond Hettinger <python@rcn.com> parents: 47615 diff changeset	109
65831 bc6cce4ab3c6 Issue 10110: Let Queue.put recognize a full queue when the maxsize parameter has been reduced. Raymond Hettinger <python@rcn.com> parents: 52259 diff changeset	110 This method is likely to be removed at some point. Use qsize() >= n
52045 aef5cfe73358 Document the suggested alternative to emtpy() and full(). Raymond Hettinger <python@rcn.com> parents: 47615 diff changeset	111 as a direct substitute, but be aware that either approach risks a race
aef5cfe73358 Document the suggested alternative to emtpy() and full(). Raymond Hettinger <python@rcn.com> parents: 47615 diff changeset	112 condition where a queue can shrink before the result of full() or
aef5cfe73358 Document the suggested alternative to emtpy() and full(). Raymond Hettinger <python@rcn.com> parents: 47615 diff changeset	113 qsize() can be used.
74307 ca2a35140e6a Make the docstring style consistent. Raymond Hettinger <python@rcn.com> parents: 74306 diff changeset	114 '''
74298 c68426b241ca Simplify the code using with-statements. Raymond Hettinger <python@rcn.com> parents: 65831 diff changeset	115 with self.mutex:
c68426b241ca Simplify the code using with-statements. Raymond Hettinger <python@rcn.com> parents: 65831 diff changeset	116 return 0 < self.maxsize <= self._qsize()
46899 bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	117
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	118 def put(self, item, block=True, timeout=None):
74307 ca2a35140e6a Make the docstring style consistent. Raymond Hettinger <python@rcn.com> parents: 74306 diff changeset	119 '''Put an item into the queue.
46899 bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	120
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	121 If optional args 'block' is true and 'timeout' is None (the default),
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	122 block if necessary until a free slot is available. If 'timeout' is
85114 2122d56d6bc5 Issue #18676: Change 'positive' to 'non-negative' in queue.py put and get Terry Jan Reedy <tjreedy@udel.edu> parents: 76597 diff changeset	123 a non-negative number, it blocks at most 'timeout' seconds and raises
46899 bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	124 the Full exception if no free slot was available within that time.
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	125 Otherwise ('block' is false), put an item on the queue if a free slot
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	126 is immediately available, else raise the Full exception ('timeout'
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	127 is ignored in that case).
74307 ca2a35140e6a Make the docstring style consistent. Raymond Hettinger <python@rcn.com> parents: 74306 diff changeset	128 '''
74298 c68426b241ca Simplify the code using with-statements. Raymond Hettinger <python@rcn.com> parents: 65831 diff changeset	129 with self.not_full:
46899 bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	130 if self.maxsize > 0:
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	131 if not block:
65831 bc6cce4ab3c6 Issue 10110: Let Queue.put recognize a full queue when the maxsize parameter has been reduced. Raymond Hettinger <python@rcn.com> parents: 52259 diff changeset	132 if self._qsize() >= self.maxsize:
46899 bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	133 raise Full
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	134 elif timeout is None:
65831 bc6cce4ab3c6 Issue 10110: Let Queue.put recognize a full queue when the maxsize parameter has been reduced. Raymond Hettinger <python@rcn.com> parents: 52259 diff changeset	135 while self._qsize() >= self.maxsize:
46899 bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	136 self.not_full.wait()
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	137 elif timeout < 0:
85114 2122d56d6bc5 Issue #18676: Change 'positive' to 'non-negative' in queue.py put and get Terry Jan Reedy <tjreedy@udel.edu> parents: 76597 diff changeset	138 raise ValueError("'timeout' must be a non-negative number")
46899 bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	139 else:
74305 190ad17f5a87 Minor code clean-ups and beautifications. Raymond Hettinger <python@rcn.com> parents: 74298 diff changeset	140 endtime = time() + timeout
65831 bc6cce4ab3c6 Issue 10110: Let Queue.put recognize a full queue when the maxsize parameter has been reduced. Raymond Hettinger <python@rcn.com> parents: 52259 diff changeset	141 while self._qsize() >= self.maxsize:
74305 190ad17f5a87 Minor code clean-ups and beautifications. Raymond Hettinger <python@rcn.com> parents: 74298 diff changeset	142 remaining = endtime - time()
46899 bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	143 if remaining <= 0.0:
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	144 raise Full
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	145 self.not_full.wait(remaining)
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	146 self._put(item)
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	147 self.unfinished_tasks += 1
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	148 self.not_empty.notify()
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	149
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	150 def get(self, block=True, timeout=None):
74307 ca2a35140e6a Make the docstring style consistent. Raymond Hettinger <python@rcn.com> parents: 74306 diff changeset	151 '''Remove and return an item from the queue.
46899 bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	152
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	153 If optional args 'block' is true and 'timeout' is None (the default),
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	154 block if necessary until an item is available. If 'timeout' is
85114 2122d56d6bc5 Issue #18676: Change 'positive' to 'non-negative' in queue.py put and get Terry Jan Reedy <tjreedy@udel.edu> parents: 76597 diff changeset	155 a non-negative number, it blocks at most 'timeout' seconds and raises
46899 bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	156 the Empty exception if no item was available within that time.
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	157 Otherwise ('block' is false), return an item if one is immediately
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	158 available, else raise the Empty exception ('timeout' is ignored
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	159 in that case).
74307 ca2a35140e6a Make the docstring style consistent. Raymond Hettinger <python@rcn.com> parents: 74306 diff changeset	160 '''
74298 c68426b241ca Simplify the code using with-statements. Raymond Hettinger <python@rcn.com> parents: 65831 diff changeset	161 with self.not_empty:
46899 bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	162 if not block:
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	163 if not self._qsize():
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	164 raise Empty
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	165 elif timeout is None:
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	166 while not self._qsize():
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	167 self.not_empty.wait()
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	168 elif timeout < 0:
85114 2122d56d6bc5 Issue #18676: Change 'positive' to 'non-negative' in queue.py put and get Terry Jan Reedy <tjreedy@udel.edu> parents: 76597 diff changeset	169 raise ValueError("'timeout' must be a non-negative number")
46899 bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	170 else:
74305 190ad17f5a87 Minor code clean-ups and beautifications. Raymond Hettinger <python@rcn.com> parents: 74298 diff changeset	171 endtime = time() + timeout
46899 bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	172 while not self._qsize():
74305 190ad17f5a87 Minor code clean-ups and beautifications. Raymond Hettinger <python@rcn.com> parents: 74298 diff changeset	173 remaining = endtime - time()
46899 bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	174 if remaining <= 0.0:
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	175 raise Empty
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	176 self.not_empty.wait(remaining)
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	177 item = self._get()
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	178 self.not_full.notify()
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	179 return item
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	180
74306 07608e67bc01 Improve clarity with keyword argument for block. Move nowait methods together. Raymond Hettinger <python@rcn.com> parents: 74305 diff changeset	181 def put_nowait(self, item):
74307 ca2a35140e6a Make the docstring style consistent. Raymond Hettinger <python@rcn.com> parents: 74306 diff changeset	182 '''Put an item into the queue without blocking.
74306 07608e67bc01 Improve clarity with keyword argument for block. Move nowait methods together. Raymond Hettinger <python@rcn.com> parents: 74305 diff changeset	183
07608e67bc01 Improve clarity with keyword argument for block. Move nowait methods together. Raymond Hettinger <python@rcn.com> parents: 74305 diff changeset	184 Only enqueue the item if a free slot is immediately available.
07608e67bc01 Improve clarity with keyword argument for block. Move nowait methods together. Raymond Hettinger <python@rcn.com> parents: 74305 diff changeset	185 Otherwise raise the Full exception.
74307 ca2a35140e6a Make the docstring style consistent. Raymond Hettinger <python@rcn.com> parents: 74306 diff changeset	186 '''
74306 07608e67bc01 Improve clarity with keyword argument for block. Move nowait methods together. Raymond Hettinger <python@rcn.com> parents: 74305 diff changeset	187 return self.put(item, block=False)
07608e67bc01 Improve clarity with keyword argument for block. Move nowait methods together. Raymond Hettinger <python@rcn.com> parents: 74305 diff changeset	188
46899 bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	189 def get_nowait(self):
74307 ca2a35140e6a Make the docstring style consistent. Raymond Hettinger <python@rcn.com> parents: 74306 diff changeset	190 '''Remove and return an item from the queue without blocking.
46899 bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	191
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	192 Only get an item if one is immediately available. Otherwise
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	193 raise the Empty exception.
74307 ca2a35140e6a Make the docstring style consistent. Raymond Hettinger <python@rcn.com> parents: 74306 diff changeset	194 '''
74306 07608e67bc01 Improve clarity with keyword argument for block. Move nowait methods together. Raymond Hettinger <python@rcn.com> parents: 74305 diff changeset	195 return self.get(block=False)
46899 bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	196
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	197 # Override these methods to implement other queue organizations
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	198 # (e.g. stack or priority queue).
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	199 # These will only be called with appropriate locks held
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	200
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	201 # Initialize the queue representation
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	202 def _init(self, maxsize):
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	203 self.queue = deque()
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	204
74305 190ad17f5a87 Minor code clean-ups and beautifications. Raymond Hettinger <python@rcn.com> parents: 74298 diff changeset	205 def _qsize(self):
46899 bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	206 return len(self.queue)
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	207
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	208 # Put a new item in the queue
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	209 def _put(self, item):
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	210 self.queue.append(item)
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	211
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	212 # Get an item from the queue
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	213 def _get(self):
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	214 return self.queue.popleft()
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	215
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	216
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	217 class PriorityQueue(Queue):
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	218 '''Variant of Queue that retrieves open entries in priority order (lowest first).
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	219
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	220 Entries are typically tuples of the form: (priority number, data).
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	221 '''
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	222
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	223 def _init(self, maxsize):
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	224 self.queue = []
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	225
74305 190ad17f5a87 Minor code clean-ups and beautifications. Raymond Hettinger <python@rcn.com> parents: 74298 diff changeset	226 def _qsize(self):
46899 bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	227 return len(self.queue)
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	228
74305 190ad17f5a87 Minor code clean-ups and beautifications. Raymond Hettinger <python@rcn.com> parents: 74298 diff changeset	229 def _put(self, item):
46899 bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	230 heappush(self.queue, item)
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	231
74305 190ad17f5a87 Minor code clean-ups and beautifications. Raymond Hettinger <python@rcn.com> parents: 74298 diff changeset	232 def _get(self):
46899 bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	233 return heappop(self.queue)
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	234
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	235
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	236 class LifoQueue(Queue):
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	237 '''Variant of Queue that retrieves most recently added entries first.'''
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	238
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	239 def _init(self, maxsize):
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	240 self.queue = []
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	241
74305 190ad17f5a87 Minor code clean-ups and beautifications. Raymond Hettinger <python@rcn.com> parents: 74298 diff changeset	242 def _qsize(self):
46899 bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	243 return len(self.queue)
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	244
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	245 def _put(self, item):
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	246 self.queue.append(item)
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	247
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	248 def _get(self):
bd2c95c165ea Rename Queue module to queue. Alexandre Vassalotti <alexandre@peadrop.com> parents: diff changeset	249 return self.queue.pop()

Mercurial > cpython-lru_cache-weakref

annotate Lib/queue.py @ 87690:66c1c9f32567 default tip