Mercurial > pygame

annotate lib/threads/__init__.py @ 3172:d1b367fc08bc tests-returncode

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Use a return code of 1 when the tests fail.
author Thomas Kluyver <takowl@gmail.com>
date Wed, 09 Jan 2013 10:13:57 +0000
parents 7cec8bcc2b75
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
1167
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
1 """
1203
c308d41a6e12 some tests for pygame.threads
akalias
parents: 1167
diff changeset
2 * Experimental *
c308d41a6e12 some tests for pygame.threads
akalias
parents: 1167
diff changeset
3
1167
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
4 Like the map function, but can use a pool of threads.
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
5
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
6 Really easy to use threads. eg. tmap(f, alist)
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
7
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
8 If you know how to use the map function, you can use threads.
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
9 """
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
10
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
11 __author__ = "Rene Dudfield"
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
12 __version__ = "0.3.0"
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
13 __license__ = 'Python license'
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
14
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
15 import traceback, sys
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
16
1553
ebd545641c8b merging with python3 branch r 2048
Lenard Lindstrom <len-l@telus.net>
parents: 1203
diff changeset
17 from pygame.compat import geterror
ebd545641c8b merging with python3 branch r 2048
Lenard Lindstrom <len-l@telus.net>
parents: 1203
diff changeset
18
ebd545641c8b merging with python3 branch r 2048
Lenard Lindstrom <len-l@telus.net>
parents: 1203
diff changeset
19 if sys.version_info[0] == 3:
3151
7cec8bcc2b75 Make pygame.threads Python 3.x compatible, as per issue #146
Lenard Lindstrom <len-l@telus.net>
parents: 1610
diff changeset
20 from queue import Queue
1553
ebd545641c8b merging with python3 branch r 2048
Lenard Lindstrom <len-l@telus.net>
parents: 1203
diff changeset
21 from queue import Empty
ebd545641c8b merging with python3 branch r 2048
Lenard Lindstrom <len-l@telus.net>
parents: 1203
diff changeset
22 elif (sys.version_info[0] == 2 and sys.version_info[1] < 5):
1167
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
23 from Py25Queue import Queue
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
24 from Py25Queue import Empty
1203
c308d41a6e12 some tests for pygame.threads
akalias
parents: 1167
diff changeset
25 else:
c308d41a6e12 some tests for pygame.threads
akalias
parents: 1167
diff changeset
26 # use up to date version
1167
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
27 from Queue import Queue
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
28 from Queue import Empty
1203
c308d41a6e12 some tests for pygame.threads
akalias
parents: 1167
diff changeset
29
1167
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
30 import threading
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
31 Thread = threading.Thread
1203
c308d41a6e12 some tests for pygame.threads
akalias
parents: 1167
diff changeset
32
1167
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
33 STOP = object()
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
34 FINISH = object()
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
35
1203
c308d41a6e12 some tests for pygame.threads
akalias
parents: 1167
diff changeset
36 # DONE_ONE = object()
c308d41a6e12 some tests for pygame.threads
akalias
parents: 1167
diff changeset
37 # DONE_TWO = object()
1167
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
38
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
39 # a default worker queue.
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
40 _wq = None
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
41
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
42 # if we are using threads or not. This is the number of workers.
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
43 _use_workers = 0
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
44
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
45 # Set this to the maximum for the amount of Cores/CPUs
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
46 # Note, that the tests early out.
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
47 # So it should only test the best number of workers +2
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
48 MAX_WORKERS_TO_TEST = 64
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
49
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
50
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
51
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
52 def init(number_of_workers = 0):
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
53 """ Does a little test to see if threading is worth it.
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
54 Sets up a global worker queue if it's worth it.
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
55
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
56 Calling init() is not required, but is generally better to do.
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
57 """
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
58 global _wq, _use_workers
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
59
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
60 if number_of_workers:
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
61 _use_workers = number_of_workers
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
62 else:
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
63 _use_workers = benchmark_workers()
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
64
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
65 # if it is best to use zero workers, then use that.
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
66 _wq = WorkerQueue(_use_workers)
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
67
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
68
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
69
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
70
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
71 def quit():
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
72 """ cleans up everything.
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
73 """
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
74 global _wq, _use_workers
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
75 _wq.stop()
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
76 _wq = None
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
77 _use_workers = False
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
78
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
79
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
80 def benchmark_workers(a_bench_func = None, the_data = None):
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
81 """ does a little test to see if workers are at all faster.
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
82 Returns the number of workers which works best.
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
83 Takes a little bit of time to run, so you should only really call
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
84 it once.
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
85 You can pass in benchmark data, and functions if you want.
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
86 a_bench_func - f(data)
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
87 the_data - data to work on.
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
88 """
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
89 global _use_workers
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
90
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
91 #TODO: try and make this scale better with slower/faster cpus.
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
92 # first find some variables so that using 0 workers takes about 1.0 seconds.
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
93 # then go from there.
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
94
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
95
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
96 # note, this will only work with pygame 1.8rc3+
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
97 # replace the doit() and the_data with something that releases the GIL
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
98
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
99
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
100 import pygame
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
101 import pygame.transform
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
102 import time
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
103
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
104 if not a_bench_func:
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
105 def doit(x):
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
106 return pygame.transform.scale(x, (544, 576))
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
107 else:
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
108 doit = a_bench_func
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
109
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
110 if not the_data:
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
111 thedata = []
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
112 for x in range(10):
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
113 thedata.append(pygame.Surface((155,155), 0, 32))
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
114 else:
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
115 thedata = the_data
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
116
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
117 best = time.time() + 100000000
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
118 best_number = 0
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
119 last_best = -1
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
120
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
121 for num_workers in range(0, MAX_WORKERS_TO_TEST):
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
122
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
123 wq = WorkerQueue(num_workers)
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
124 t1 = time.time()
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
125 for xx in range(20):
1553
ebd545641c8b merging with python3 branch r 2048
Lenard Lindstrom <len-l@telus.net>
parents: 1203
diff changeset
126 print ("active count:%s" % threading.activeCount())
1167
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
127 results = tmap(doit, thedata, worker_queue = wq)
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
128 t2 = time.time()
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
129
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
130 wq.stop()
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
131
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
132
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
133 total_time = t2 - t1
1553
ebd545641c8b merging with python3 branch r 2048
Lenard Lindstrom <len-l@telus.net>
parents: 1203
diff changeset
134 print ("total time num_workers:%s: time:%s:" % (num_workers, total_time))
1167
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
135
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
136 if total_time < best:
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
137 last_best = best_number
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
138 best_number =num_workers
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
139 best = total_time
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
140
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
141 if num_workers - best_number > 1:
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
142 # We tried to add more, but it didn't like it.
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
143 # so we stop with testing at this number.
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
144 break
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
145
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
146
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
147 return best_number
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
148
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
149
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
150
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
151
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
152 class WorkerQueue(object):
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
153
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
154 def __init__(self, num_workers = 20):
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
155 self.queue = Queue()
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
156 self.pool = []
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
157 self._setup_workers(num_workers)
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
158
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
159 def _setup_workers(self, num_workers):
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
160 """ Sets up the worker threads
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
161 NOTE: undefined behaviour if you call this again.
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
162 """
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
163 self.pool = []
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
164
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
165 for _ in range(num_workers):
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
166 self.pool.append(Thread(target=self.threadloop))
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
167
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
168 for a_thread in self.pool:
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
169 a_thread.setDaemon(True)
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
170 a_thread.start()
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
171
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
172
1203
c308d41a6e12 some tests for pygame.threads
akalias
parents: 1167
diff changeset
173 def do(self, f, *args, **kwArgs):
1167
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
174 """ puts a function on a queue for running later.
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
175 """
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
176 self.queue.put((f, args, kwArgs))
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
177
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
178
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
179 def stop(self):
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
180 """ Stops the WorkerQueue, waits for all of the threads to finish up.
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
181 """
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
182 self.queue.put(STOP)
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
183 for thread in self.pool:
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
184 thread.join()
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
185
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
186
1203
c308d41a6e12 some tests for pygame.threads
akalias
parents: 1167
diff changeset
187 def threadloop(self): #, finish = False):
1167
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
188 """ Loops until all of the tasks are finished.
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
189 """
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
190 while True:
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
191 args = self.queue.get()
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
192 if args is STOP:
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
193 self.queue.put(STOP)
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
194 self.queue.task_done()
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
195 break
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
196 else:
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
197 try:
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
198 args[0](*args[1], **args[2])
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
199 finally:
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
200 # clean up the queue, raise the exception.
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
201 self.queue.task_done()
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
202 #raise
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
203
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
204
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
205 def wait(self):
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
206 """ waits until all tasks are complete.
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
207 """
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
208 self.queue.join()
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
209
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
210 class FuncResult:
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
211 """ Used for wrapping up a function call so that the results are stored
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
212 inside the instances result attribute.
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
213 """
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
214 def __init__(self, f, callback = None, errback = None):
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
215 """ f - is the function we that we call
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
216 callback(result) - this is called when the function(f) returns
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
217 errback(exception) - this is called when the function(f) raises
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
218 an exception.
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
219 """
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
220 self.f = f
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
221 self.exception = None
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
222 self.callback = callback
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
223 self.errback = errback
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
224
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
225 def __call__(self, *args, **kwargs):
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
226
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
227 #we try to call the function here. If it fails we store the exception.
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
228 try:
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
229 self.result = self.f(*args, **kwargs)
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
230 if self.callback:
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
231 self.callback(self.result)
1553
ebd545641c8b merging with python3 branch r 2048
Lenard Lindstrom <len-l@telus.net>
parents: 1203
diff changeset
232 except Exception:
ebd545641c8b merging with python3 branch r 2048
Lenard Lindstrom <len-l@telus.net>
parents: 1203
diff changeset
233 self.exception = geterror()
1167
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
234 if self.errback:
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
235 self.errback(self.exception)
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
236
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
237
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
238 def tmap(f, seq_args, num_workers = 20, worker_queue = None, wait = True, stop_on_error = True):
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
239 """ like map, but uses a thread pool to execute.
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
240 num_workers - the number of worker threads that will be used. If pool
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
241 is passed in, then the num_workers arg is ignored.
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
242 worker_queue - you can optionally pass in an existing WorkerQueue.
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
243 wait - True means that the results are returned when everything is finished.
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
244 False means that we return the [worker_queue, results] right away instead.
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
245 results, is returned as a list of FuncResult instances.
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
246 stop_on_error -
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
247 """
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
248
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
249 if worker_queue:
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
250 wq = worker_queue
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
251 else:
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
252 # see if we have a global queue to work with.
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
253 if _wq:
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
254 wq = _wq
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
255 else:
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
256 if num_workers == 0:
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
257 return map(f, seq_args)
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
258
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
259 wq = WorkerQueue(num_workers)
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
260
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
261 # we short cut it here if the number of workers is 0.
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
262 # normal map should be faster in this case.
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
263 if len(wq.pool) == 0:
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
264 return map(f, seq_args)
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
265
1553
ebd545641c8b merging with python3 branch r 2048
Lenard Lindstrom <len-l@telus.net>
parents: 1203
diff changeset
266 #print ("queue size:%s" % wq.queue.qsize())
1167
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
267
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
268
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
269 #TODO: divide the data (seq_args) into even chunks and
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
270 # then pass each thread a map(f, equal_part(seq_args))
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
271 # That way there should be less locking, and overhead.
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
272
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
273
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
274
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
275 results = []
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
276 for sa in seq_args:
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
277 results.append(FuncResult(f))
1203
c308d41a6e12 some tests for pygame.threads
akalias
parents: 1167
diff changeset
278 wq.do(results[-1], sa)
1167
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
279
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
280
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
281 #wq.stop()
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
282
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
283 if wait:
1553
ebd545641c8b merging with python3 branch r 2048
Lenard Lindstrom <len-l@telus.net>
parents: 1203
diff changeset
284 #print ("wait")
1167
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
285 wq.wait()
1553
ebd545641c8b merging with python3 branch r 2048
Lenard Lindstrom <len-l@telus.net>
parents: 1203
diff changeset
286 #print ("after wait")
ebd545641c8b merging with python3 branch r 2048
Lenard Lindstrom <len-l@telus.net>
parents: 1203
diff changeset
287 #print ("queue size:%s" % wq.queue.qsize())
1167
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
288 if wq.queue.qsize():
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
289 raise Exception("buggy threadmap")
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
290 # if we created a worker queue, we need to stop it.
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
291 if not worker_queue and not _wq:
1553
ebd545641c8b merging with python3 branch r 2048
Lenard Lindstrom <len-l@telus.net>
parents: 1203
diff changeset
292 #print ("stoping")
1167
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
293 wq.stop()
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
294 if wq.queue.qsize():
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
295 um = wq.queue.get()
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
296 if not um is STOP:
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
297 raise Exception("buggy threadmap")
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
298
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
299
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
300 # see if there were any errors. If so raise the first one. This matches map behaviour.
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
301 # TODO: the traceback doesn't show up nicely.
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
302 # NOTE: TODO: we might want to return the results anyway? This should be an option.
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
303 if stop_on_error:
3151
7cec8bcc2b75 Make pygame.threads Python 3.x compatible, as per issue #146
Lenard Lindstrom <len-l@telus.net>
parents: 1610
diff changeset
304 error_ones = list(filter(lambda x:x.exception, results))
1167
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
305 if error_ones:
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
306 raise error_ones[0].exception
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
307
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
308 return map(lambda x:x.result, results)
4bc6d1bf7ceb Moved pygame.threads into it's own package due to Queue needing to import built-in time module, was conflicting with time.pyd in same dir.
akalias
parents:
diff changeset
309 else:
1610
6c5040262b53 implement *_tags.py module replacement to *_test.py level tags.
Lenard Lindstrom <len-l@telus.net>
parents: 1553
diff changeset
310 return [wq, results]