cpython-lru_cache-weakref: Lib/test/test

annotate Lib/test/test_threading.py @ 80661:94de77bd0b4b 2.7

Issue #16194: document imp.load_dynamic problems when read several extension modules from the same binary object file.

author	Andrew Svetlov <andrew.svetlov@gmail.com>
date	Thu, 29 Nov 2012 19:51:28 +0200
parents	ab9d6c4907e7
children	ba54011aa295

rev	line source
19405 9de83dda9f09 add a few test cases for threading module. Skip Montanaro <skip@pobox.com> parents: diff changeset	1 # Very rudimentary test of threading module
9de83dda9f09 add a few test cases for threading module. Skip Montanaro <skip@pobox.com> parents: diff changeset	2
34362 cc27b249755d Converted to a unittest. Added checks that the bounded semaphore actually Tim Peters <tim.peters@gmail.com> parents: 24364 diff changeset	3 import test.test_support
24364 aa0d2d51fbde Get rid of relative imports in all unittests. Now anything that Barry Warsaw <barry@python.org> parents: 19413 diff changeset	4 from test.test_support import verbose
76420 ab9d6c4907e7 Issue #14308: Fix an exception when a "dummy" thread is in the threading module's active list after a fork(). Antoine Pitrou <solipsis@pitrou.net> parents: 74829 diff changeset	5 from test.script_helper import assert_python_ok
ab9d6c4907e7 Issue #14308: Fix an exception when a "dummy" thread is in the threading module's active list after a fork(). Antoine Pitrou <solipsis@pitrou.net> parents: 74829 diff changeset	6
19405 9de83dda9f09 add a few test cases for threading module. Skip Montanaro <skip@pobox.com> parents: diff changeset	7 import random
47438 7eaaad09b6cd Adds a Thread.getIdent() method to provide the _get_ident() value for Gregory P. Smith <greg@mad-scientist.com> parents: 46280 diff changeset	8 import re
44871 ffacb44673ea Backport of r60190: Gregory P. Smith <greg@mad-scientist.com> parents: 39434 diff changeset	9 import sys
61046 3a0484200034 Issue #7449, last part (11): fix many tests if thread support is disabled Victor Stinner <victor.stinner@haypocalc.com> parents: 59219 diff changeset	10 thread = test.test_support.import_module('thread')
3a0484200034 Issue #7449, last part (11): fix many tests if thread support is disabled Victor Stinner <victor.stinner@haypocalc.com> parents: 59219 diff changeset	11 threading = test.test_support.import_module('threading')
19405 9de83dda9f09 add a few test cases for threading module. Skip Montanaro <skip@pobox.com> parents: diff changeset	12 import time
34362 cc27b249755d Converted to a unittest. Added checks that the bounded semaphore actually Tim Peters <tim.peters@gmail.com> parents: 24364 diff changeset	13 import unittest
45477 241cf81a0508 Prevent classes like: Jeffrey Yasskin <jyasskin@gmail.com> parents: 44870 diff changeset	14 import weakref
67341 3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	15 import os
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	16 import subprocess
19405 9de83dda9f09 add a few test cases for threading module. Skip Montanaro <skip@pobox.com> parents: diff changeset	17
57159 0eb56a8e6262 Merged revisions 76137,76172 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 57140 diff changeset	18 from test import lock_tests
19405 9de83dda9f09 add a few test cases for threading module. Skip Montanaro <skip@pobox.com> parents: diff changeset	19
34362 cc27b249755d Converted to a unittest. Added checks that the bounded semaphore actually Tim Peters <tim.peters@gmail.com> parents: 24364 diff changeset	20 # A trivial mutable counter.
cc27b249755d Converted to a unittest. Added checks that the bounded semaphore actually Tim Peters <tim.peters@gmail.com> parents: 24364 diff changeset	21 class Counter(object):
cc27b249755d Converted to a unittest. Added checks that the bounded semaphore actually Tim Peters <tim.peters@gmail.com> parents: 24364 diff changeset	22 def __init__(self):
cc27b249755d Converted to a unittest. Added checks that the bounded semaphore actually Tim Peters <tim.peters@gmail.com> parents: 24364 diff changeset	23 self.value = 0
cc27b249755d Converted to a unittest. Added checks that the bounded semaphore actually Tim Peters <tim.peters@gmail.com> parents: 24364 diff changeset	24 def inc(self):
cc27b249755d Converted to a unittest. Added checks that the bounded semaphore actually Tim Peters <tim.peters@gmail.com> parents: 24364 diff changeset	25 self.value += 1
cc27b249755d Converted to a unittest. Added checks that the bounded semaphore actually Tim Peters <tim.peters@gmail.com> parents: 24364 diff changeset	26 def dec(self):
cc27b249755d Converted to a unittest. Added checks that the bounded semaphore actually Tim Peters <tim.peters@gmail.com> parents: 24364 diff changeset	27 self.value -= 1
cc27b249755d Converted to a unittest. Added checks that the bounded semaphore actually Tim Peters <tim.peters@gmail.com> parents: 24364 diff changeset	28 def get(self):
cc27b249755d Converted to a unittest. Added checks that the bounded semaphore actually Tim Peters <tim.peters@gmail.com> parents: 24364 diff changeset	29 return self.value
19405 9de83dda9f09 add a few test cases for threading module. Skip Montanaro <skip@pobox.com> parents: diff changeset	30
9de83dda9f09 add a few test cases for threading module. Skip Montanaro <skip@pobox.com> parents: diff changeset	31 class TestThread(threading.Thread):
34362 cc27b249755d Converted to a unittest. Added checks that the bounded semaphore actually Tim Peters <tim.peters@gmail.com> parents: 24364 diff changeset	32 def __init__(self, name, testcase, sema, mutex, nrunning):
cc27b249755d Converted to a unittest. Added checks that the bounded semaphore actually Tim Peters <tim.peters@gmail.com> parents: 24364 diff changeset	33 threading.Thread.__init__(self, name=name)
cc27b249755d Converted to a unittest. Added checks that the bounded semaphore actually Tim Peters <tim.peters@gmail.com> parents: 24364 diff changeset	34 self.testcase = testcase
cc27b249755d Converted to a unittest. Added checks that the bounded semaphore actually Tim Peters <tim.peters@gmail.com> parents: 24364 diff changeset	35 self.sema = sema
cc27b249755d Converted to a unittest. Added checks that the bounded semaphore actually Tim Peters <tim.peters@gmail.com> parents: 24364 diff changeset	36 self.mutex = mutex
cc27b249755d Converted to a unittest. Added checks that the bounded semaphore actually Tim Peters <tim.peters@gmail.com> parents: 24364 diff changeset	37 self.nrunning = nrunning
cc27b249755d Converted to a unittest. Added checks that the bounded semaphore actually Tim Peters <tim.peters@gmail.com> parents: 24364 diff changeset	38
19405 9de83dda9f09 add a few test cases for threading module. Skip Montanaro <skip@pobox.com> parents: diff changeset	39 def run(self):
45975 8ef05aceef2b Speed test_threading up from 14s to .5s, and avoid a deadlock on certain Jeffrey Yasskin <jyasskin@gmail.com> parents: 45478 diff changeset	40 delay = random.random() / 10000.0
19405 9de83dda9f09 add a few test cases for threading module. Skip Montanaro <skip@pobox.com> parents: diff changeset	41 if verbose:
45975 8ef05aceef2b Speed test_threading up from 14s to .5s, and avoid a deadlock on certain Jeffrey Yasskin <jyasskin@gmail.com> parents: 45478 diff changeset	42 print 'task %s will run for %.1f usec' % (
48288 e143a8f35241 backport threading property changes Benjamin Peterson <benjamin@python.org> parents: 48287 diff changeset	43 self.name, delay * 1e6)
34362 cc27b249755d Converted to a unittest. Added checks that the bounded semaphore actually Tim Peters <tim.peters@gmail.com> parents: 24364 diff changeset	44
45975 8ef05aceef2b Speed test_threading up from 14s to .5s, and avoid a deadlock on certain Jeffrey Yasskin <jyasskin@gmail.com> parents: 45478 diff changeset	45 with self.sema:
8ef05aceef2b Speed test_threading up from 14s to .5s, and avoid a deadlock on certain Jeffrey Yasskin <jyasskin@gmail.com> parents: 45478 diff changeset	46 with self.mutex:
8ef05aceef2b Speed test_threading up from 14s to .5s, and avoid a deadlock on certain Jeffrey Yasskin <jyasskin@gmail.com> parents: 45478 diff changeset	47 self.nrunning.inc()
8ef05aceef2b Speed test_threading up from 14s to .5s, and avoid a deadlock on certain Jeffrey Yasskin <jyasskin@gmail.com> parents: 45478 diff changeset	48 if verbose:
8ef05aceef2b Speed test_threading up from 14s to .5s, and avoid a deadlock on certain Jeffrey Yasskin <jyasskin@gmail.com> parents: 45478 diff changeset	49 print self.nrunning.get(), 'tasks are running'
55002 6768b2350c57 convert usage of fail* to assert* Benjamin Peterson <benjamin@python.org> parents: 52593 diff changeset	50 self.testcase.assertTrue(self.nrunning.get() <= 3)
34362 cc27b249755d Converted to a unittest. Added checks that the bounded semaphore actually Tim Peters <tim.peters@gmail.com> parents: 24364 diff changeset	51
45975 8ef05aceef2b Speed test_threading up from 14s to .5s, and avoid a deadlock on certain Jeffrey Yasskin <jyasskin@gmail.com> parents: 45478 diff changeset	52 time.sleep(delay)
8ef05aceef2b Speed test_threading up from 14s to .5s, and avoid a deadlock on certain Jeffrey Yasskin <jyasskin@gmail.com> parents: 45478 diff changeset	53 if verbose:
48288 e143a8f35241 backport threading property changes Benjamin Peterson <benjamin@python.org> parents: 48287 diff changeset	54 print 'task', self.name, 'done'
34362 cc27b249755d Converted to a unittest. Added checks that the bounded semaphore actually Tim Peters <tim.peters@gmail.com> parents: 24364 diff changeset	55
45975 8ef05aceef2b Speed test_threading up from 14s to .5s, and avoid a deadlock on certain Jeffrey Yasskin <jyasskin@gmail.com> parents: 45478 diff changeset	56 with self.mutex:
8ef05aceef2b Speed test_threading up from 14s to .5s, and avoid a deadlock on certain Jeffrey Yasskin <jyasskin@gmail.com> parents: 45478 diff changeset	57 self.nrunning.dec()
55002 6768b2350c57 convert usage of fail* to assert* Benjamin Peterson <benjamin@python.org> parents: 52593 diff changeset	58 self.testcase.assertTrue(self.nrunning.get() >= 0)
45975 8ef05aceef2b Speed test_threading up from 14s to .5s, and avoid a deadlock on certain Jeffrey Yasskin <jyasskin@gmail.com> parents: 45478 diff changeset	59 if verbose:
8ef05aceef2b Speed test_threading up from 14s to .5s, and avoid a deadlock on certain Jeffrey Yasskin <jyasskin@gmail.com> parents: 45478 diff changeset	60 print '%s is finished. %d tasks are running' % (
48288 e143a8f35241 backport threading property changes Benjamin Peterson <benjamin@python.org> parents: 48287 diff changeset	61 self.name, self.nrunning.get())
34362 cc27b249755d Converted to a unittest. Added checks that the bounded semaphore actually Tim Peters <tim.peters@gmail.com> parents: 24364 diff changeset	62
56882 5f61bd3aa427 Suppress transient refleaks in test_threading. Antoine Pitrou <solipsis@pitrou.net> parents: 56686 diff changeset	63 class BaseTestCase(unittest.TestCase):
5f61bd3aa427 Suppress transient refleaks in test_threading. Antoine Pitrou <solipsis@pitrou.net> parents: 56686 diff changeset	64 def setUp(self):
5f61bd3aa427 Suppress transient refleaks in test_threading. Antoine Pitrou <solipsis@pitrou.net> parents: 56686 diff changeset	65 self._threads = test.test_support.threading_setup()
5f61bd3aa427 Suppress transient refleaks in test_threading. Antoine Pitrou <solipsis@pitrou.net> parents: 56686 diff changeset	66
5f61bd3aa427 Suppress transient refleaks in test_threading. Antoine Pitrou <solipsis@pitrou.net> parents: 56686 diff changeset	67 def tearDown(self):
5f61bd3aa427 Suppress transient refleaks in test_threading. Antoine Pitrou <solipsis@pitrou.net> parents: 56686 diff changeset	68 test.test_support.threading_cleanup(*self._threads)
5f61bd3aa427 Suppress transient refleaks in test_threading. Antoine Pitrou <solipsis@pitrou.net> parents: 56686 diff changeset	69 test.test_support.reap_children()
5f61bd3aa427 Suppress transient refleaks in test_threading. Antoine Pitrou <solipsis@pitrou.net> parents: 56686 diff changeset	70
5f61bd3aa427 Suppress transient refleaks in test_threading. Antoine Pitrou <solipsis@pitrou.net> parents: 56686 diff changeset	71
5f61bd3aa427 Suppress transient refleaks in test_threading. Antoine Pitrou <solipsis@pitrou.net> parents: 56686 diff changeset	72 class ThreadTests(BaseTestCase):
34362 cc27b249755d Converted to a unittest. Added checks that the bounded semaphore actually Tim Peters <tim.peters@gmail.com> parents: 24364 diff changeset	73
cc27b249755d Converted to a unittest. Added checks that the bounded semaphore actually Tim Peters <tim.peters@gmail.com> parents: 24364 diff changeset	74 # Create a bunch of threads, let each do some work, wait until all are
cc27b249755d Converted to a unittest. Added checks that the bounded semaphore actually Tim Peters <tim.peters@gmail.com> parents: 24364 diff changeset	75 # done.
cc27b249755d Converted to a unittest. Added checks that the bounded semaphore actually Tim Peters <tim.peters@gmail.com> parents: 24364 diff changeset	76 def test_various_ops(self):
cc27b249755d Converted to a unittest. Added checks that the bounded semaphore actually Tim Peters <tim.peters@gmail.com> parents: 24364 diff changeset	77 # This takes about n/3 seconds to run (about n/3 clumps of tasks,
cc27b249755d Converted to a unittest. Added checks that the bounded semaphore actually Tim Peters <tim.peters@gmail.com> parents: 24364 diff changeset	78 # times about 1 second per clump).
cc27b249755d Converted to a unittest. Added checks that the bounded semaphore actually Tim Peters <tim.peters@gmail.com> parents: 24364 diff changeset	79 NUMTASKS = 10
19405 9de83dda9f09 add a few test cases for threading module. Skip Montanaro <skip@pobox.com> parents: diff changeset	80
34362 cc27b249755d Converted to a unittest. Added checks that the bounded semaphore actually Tim Peters <tim.peters@gmail.com> parents: 24364 diff changeset	81 # no more than 3 of the 10 can run at once
cc27b249755d Converted to a unittest. Added checks that the bounded semaphore actually Tim Peters <tim.peters@gmail.com> parents: 24364 diff changeset	82 sema = threading.BoundedSemaphore(value=3)
cc27b249755d Converted to a unittest. Added checks that the bounded semaphore actually Tim Peters <tim.peters@gmail.com> parents: 24364 diff changeset	83 mutex = threading.RLock()
cc27b249755d Converted to a unittest. Added checks that the bounded semaphore actually Tim Peters <tim.peters@gmail.com> parents: 24364 diff changeset	84 numrunning = Counter()
cc27b249755d Converted to a unittest. Added checks that the bounded semaphore actually Tim Peters <tim.peters@gmail.com> parents: 24364 diff changeset	85
cc27b249755d Converted to a unittest. Added checks that the bounded semaphore actually Tim Peters <tim.peters@gmail.com> parents: 24364 diff changeset	86 threads = []
cc27b249755d Converted to a unittest. Added checks that the bounded semaphore actually Tim Peters <tim.peters@gmail.com> parents: 24364 diff changeset	87
cc27b249755d Converted to a unittest. Added checks that the bounded semaphore actually Tim Peters <tim.peters@gmail.com> parents: 24364 diff changeset	88 for i in range(NUMTASKS):
cc27b249755d Converted to a unittest. Added checks that the bounded semaphore actually Tim Peters <tim.peters@gmail.com> parents: 24364 diff changeset	89 t = TestThread("<thread %d>"%i, self, sema, mutex, numrunning)
cc27b249755d Converted to a unittest. Added checks that the bounded semaphore actually Tim Peters <tim.peters@gmail.com> parents: 24364 diff changeset	90 threads.append(t)
55002 6768b2350c57 convert usage of fail* to assert* Benjamin Peterson <benjamin@python.org> parents: 52593 diff changeset	91 self.assertEqual(t.ident, None)
6768b2350c57 convert usage of fail* to assert* Benjamin Peterson <benjamin@python.org> parents: 52593 diff changeset	92 self.assertTrue(re.match('<TestThread\(.*, initial\)>', repr(t)))
34362 cc27b249755d Converted to a unittest. Added checks that the bounded semaphore actually Tim Peters <tim.peters@gmail.com> parents: 24364 diff changeset	93 t.start()
19405 9de83dda9f09 add a few test cases for threading module. Skip Montanaro <skip@pobox.com> parents: diff changeset	94
34362 cc27b249755d Converted to a unittest. Added checks that the bounded semaphore actually Tim Peters <tim.peters@gmail.com> parents: 24364 diff changeset	95 if verbose:
cc27b249755d Converted to a unittest. Added checks that the bounded semaphore actually Tim Peters <tim.peters@gmail.com> parents: 24364 diff changeset	96 print 'waiting for all tasks to complete'
cc27b249755d Converted to a unittest. Added checks that the bounded semaphore actually Tim Peters <tim.peters@gmail.com> parents: 24364 diff changeset	97 for t in threads:
34363 35f31790c6f9 threading._DummyThread.__init__(): document obscure new code. Tim Peters <tim.peters@gmail.com> parents: 34362 diff changeset	98 t.join(NUMTASKS)
55002 6768b2350c57 convert usage of fail* to assert* Benjamin Peterson <benjamin@python.org> parents: 52593 diff changeset	99 self.assertTrue(not t.is_alive())
6768b2350c57 convert usage of fail* to assert* Benjamin Peterson <benjamin@python.org> parents: 52593 diff changeset	100 self.assertNotEqual(t.ident, 0)
52596 21ba0b3fe819 Merged revisions 70897 via svnmerge from Benjamin Peterson <benjamin@python.org> parents: 49361 diff changeset	101 self.assertFalse(t.ident is None)
55002 6768b2350c57 convert usage of fail* to assert* Benjamin Peterson <benjamin@python.org> parents: 52593 diff changeset	102 self.assertTrue(re.match('<TestThread\(.*, \w+ -?\d+\)>', repr(t)))
34362 cc27b249755d Converted to a unittest. Added checks that the bounded semaphore actually Tim Peters <tim.peters@gmail.com> parents: 24364 diff changeset	103 if verbose:
cc27b249755d Converted to a unittest. Added checks that the bounded semaphore actually Tim Peters <tim.peters@gmail.com> parents: 24364 diff changeset	104 print 'all tasks done'
cc27b249755d Converted to a unittest. Added checks that the bounded semaphore actually Tim Peters <tim.peters@gmail.com> parents: 24364 diff changeset	105 self.assertEqual(numrunning.get(), 0)
19405 9de83dda9f09 add a few test cases for threading module. Skip Montanaro <skip@pobox.com> parents: diff changeset	106
52596 21ba0b3fe819 Merged revisions 70897 via svnmerge from Benjamin Peterson <benjamin@python.org> parents: 49361 diff changeset	107 def test_ident_of_no_threading_threads(self):
21ba0b3fe819 Merged revisions 70897 via svnmerge from Benjamin Peterson <benjamin@python.org> parents: 49361 diff changeset	108 # The ident still must work for the main thread and dummy threads.
21ba0b3fe819 Merged revisions 70897 via svnmerge from Benjamin Peterson <benjamin@python.org> parents: 49361 diff changeset	109 self.assertFalse(threading.currentThread().ident is None)
21ba0b3fe819 Merged revisions 70897 via svnmerge from Benjamin Peterson <benjamin@python.org> parents: 49361 diff changeset	110 def f():
21ba0b3fe819 Merged revisions 70897 via svnmerge from Benjamin Peterson <benjamin@python.org> parents: 49361 diff changeset	111 ident.append(threading.currentThread().ident)
21ba0b3fe819 Merged revisions 70897 via svnmerge from Benjamin Peterson <benjamin@python.org> parents: 49361 diff changeset	112 done.set()
21ba0b3fe819 Merged revisions 70897 via svnmerge from Benjamin Peterson <benjamin@python.org> parents: 49361 diff changeset	113 done = threading.Event()
21ba0b3fe819 Merged revisions 70897 via svnmerge from Benjamin Peterson <benjamin@python.org> parents: 49361 diff changeset	114 ident = []
21ba0b3fe819 Merged revisions 70897 via svnmerge from Benjamin Peterson <benjamin@python.org> parents: 49361 diff changeset	115 thread.start_new_thread(f, ())
21ba0b3fe819 Merged revisions 70897 via svnmerge from Benjamin Peterson <benjamin@python.org> parents: 49361 diff changeset	116 done.wait()
21ba0b3fe819 Merged revisions 70897 via svnmerge from Benjamin Peterson <benjamin@python.org> parents: 49361 diff changeset	117 self.assertFalse(ident[0] is None)
57140 3a0e69001215 Merged revisions 76148 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 56781 diff changeset	118 # Kill the "immortal" _DummyThread
3a0e69001215 Merged revisions 76148 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 56781 diff changeset	119 del threading._active[ident[0]]
52596 21ba0b3fe819 Merged revisions 70897 via svnmerge from Benjamin Peterson <benjamin@python.org> parents: 49361 diff changeset	120
38802 6e0c0c7cf068 Increase the small thread stack size to get the test Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 38794 diff changeset	121 # run with a small(ish) thread stack size (256kB)
38794 f7d23bca599f Patch #1454481: Make thread stack size runtime tunable. Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 38592 diff changeset	122 def test_various_ops_small_stack(self):
f7d23bca599f Patch #1454481: Make thread stack size runtime tunable. Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 38592 diff changeset	123 if verbose:
38802 6e0c0c7cf068 Increase the small thread stack size to get the test Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 38794 diff changeset	124 print 'with 256kB thread stack size...'
39389 fc101cc354a7 test_threading now skips testing alternate thread stack sizes on Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 38802 diff changeset	125 try:
fc101cc354a7 test_threading now skips testing alternate thread stack sizes on Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 38802 diff changeset	126 threading.stack_size(262144)
fc101cc354a7 test_threading now skips testing alternate thread stack sizes on Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 38802 diff changeset	127 except thread.error:
fc101cc354a7 test_threading now skips testing alternate thread stack sizes on Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 38802 diff changeset	128 if verbose:
fc101cc354a7 test_threading now skips testing alternate thread stack sizes on Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 38802 diff changeset	129 print 'platform does not support changing thread stack size'
fc101cc354a7 test_threading now skips testing alternate thread stack sizes on Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 38802 diff changeset	130 return
38794 f7d23bca599f Patch #1454481: Make thread stack size runtime tunable. Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 38592 diff changeset	131 self.test_various_ops()
f7d23bca599f Patch #1454481: Make thread stack size runtime tunable. Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 38592 diff changeset	132 threading.stack_size(0)
f7d23bca599f Patch #1454481: Make thread stack size runtime tunable. Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 38592 diff changeset	133
f7d23bca599f Patch #1454481: Make thread stack size runtime tunable. Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 38592 diff changeset	134 # run with a large thread stack size (1MB)
f7d23bca599f Patch #1454481: Make thread stack size runtime tunable. Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 38592 diff changeset	135 def test_various_ops_large_stack(self):
f7d23bca599f Patch #1454481: Make thread stack size runtime tunable. Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 38592 diff changeset	136 if verbose:
f7d23bca599f Patch #1454481: Make thread stack size runtime tunable. Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 38592 diff changeset	137 print 'with 1MB thread stack size...'
39389 fc101cc354a7 test_threading now skips testing alternate thread stack sizes on Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 38802 diff changeset	138 try:
fc101cc354a7 test_threading now skips testing alternate thread stack sizes on Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 38802 diff changeset	139 threading.stack_size(0x100000)
fc101cc354a7 test_threading now skips testing alternate thread stack sizes on Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 38802 diff changeset	140 except thread.error:
fc101cc354a7 test_threading now skips testing alternate thread stack sizes on Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 38802 diff changeset	141 if verbose:
fc101cc354a7 test_threading now skips testing alternate thread stack sizes on Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 38802 diff changeset	142 print 'platform does not support changing thread stack size'
fc101cc354a7 test_threading now skips testing alternate thread stack sizes on Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 38802 diff changeset	143 return
38794 f7d23bca599f Patch #1454481: Make thread stack size runtime tunable. Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 38592 diff changeset	144 self.test_various_ops()
f7d23bca599f Patch #1454481: Make thread stack size runtime tunable. Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 38592 diff changeset	145 threading.stack_size(0)
f7d23bca599f Patch #1454481: Make thread stack size runtime tunable. Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 38592 diff changeset	146
34363 35f31790c6f9 threading._DummyThread.__init__(): document obscure new code. Tim Peters <tim.peters@gmail.com> parents: 34362 diff changeset	147 def test_foreign_thread(self):
35f31790c6f9 threading._DummyThread.__init__(): document obscure new code. Tim Peters <tim.peters@gmail.com> parents: 34362 diff changeset	148 # Check that a "foreign" thread can use the threading module.
35f31790c6f9 threading._DummyThread.__init__(): document obscure new code. Tim Peters <tim.peters@gmail.com> parents: 34362 diff changeset	149 def f(mutex):
57159 0eb56a8e6262 Merged revisions 76137,76172 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 57140 diff changeset	150 # Calling current_thread() forces an entry for the foreign
34363 35f31790c6f9 threading._DummyThread.__init__(): document obscure new code. Tim Peters <tim.peters@gmail.com> parents: 34362 diff changeset	151 # thread to get made in the threading._active map.
57159 0eb56a8e6262 Merged revisions 76137,76172 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 57140 diff changeset	152 threading.current_thread()
34363 35f31790c6f9 threading._DummyThread.__init__(): document obscure new code. Tim Peters <tim.peters@gmail.com> parents: 34362 diff changeset	153 mutex.release()
35f31790c6f9 threading._DummyThread.__init__(): document obscure new code. Tim Peters <tim.peters@gmail.com> parents: 34362 diff changeset	154
35f31790c6f9 threading._DummyThread.__init__(): document obscure new code. Tim Peters <tim.peters@gmail.com> parents: 34362 diff changeset	155 mutex = threading.Lock()
35f31790c6f9 threading._DummyThread.__init__(): document obscure new code. Tim Peters <tim.peters@gmail.com> parents: 34362 diff changeset	156 mutex.acquire()
35f31790c6f9 threading._DummyThread.__init__(): document obscure new code. Tim Peters <tim.peters@gmail.com> parents: 34362 diff changeset	157 tid = thread.start_new_thread(f, (mutex,))
35f31790c6f9 threading._DummyThread.__init__(): document obscure new code. Tim Peters <tim.peters@gmail.com> parents: 34362 diff changeset	158 # Wait for the thread to finish.
35f31790c6f9 threading._DummyThread.__init__(): document obscure new code. Tim Peters <tim.peters@gmail.com> parents: 34362 diff changeset	159 mutex.acquire()
58460 17891566a478 use assert[Not]In where appropriate Ezio Melotti <ezio.melotti@gmail.com> parents: 57157 diff changeset	160 self.assertIn(tid, threading._active)
58472 6ffcdafa0118 use assert[Not]IsInstance where appropriate Ezio Melotti <ezio.melotti@gmail.com> parents: 58460 diff changeset	161 self.assertIsInstance(threading._active[tid], threading._DummyThread)
34363 35f31790c6f9 threading._DummyThread.__init__(): document obscure new code. Tim Peters <tim.peters@gmail.com> parents: 34362 diff changeset	162 del threading._active[tid]
34362 cc27b249755d Converted to a unittest. Added checks that the bounded semaphore actually Tim Peters <tim.peters@gmail.com> parents: 24364 diff changeset	163
39431 de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	164 # PyThreadState_SetAsyncExc() is a CPython-only gimmick, not (currently)
de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	165 # exposed at the Python level. This test relies on ctypes to get at it.
de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	166 def test_PyThreadState_SetAsyncExc(self):
de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	167 try:
de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	168 import ctypes
de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	169 except ImportError:
de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	170 if verbose:
de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	171 print "test_PyThreadState_SetAsyncExc can't import ctypes"
de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	172 return # can't do anything
de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	173
de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	174 set_async_exc = ctypes.pythonapi.PyThreadState_SetAsyncExc
de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	175
de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	176 class AsyncExc(Exception):
de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	177 pass
de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	178
de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	179 exception = ctypes.py_object(AsyncExc)
de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	180
56598 a0e41dfb972b Add a test for same-thread asynchronous exceptions (see #1779233). Antoine Pitrou <solipsis@pitrou.net> parents: 55002 diff changeset	181 # First check it works when setting the exception from the same thread.
a0e41dfb972b Add a test for same-thread asynchronous exceptions (see #1779233). Antoine Pitrou <solipsis@pitrou.net> parents: 55002 diff changeset	182 tid = thread.get_ident()
a0e41dfb972b Add a test for same-thread asynchronous exceptions (see #1779233). Antoine Pitrou <solipsis@pitrou.net> parents: 55002 diff changeset	183
a0e41dfb972b Add a test for same-thread asynchronous exceptions (see #1779233). Antoine Pitrou <solipsis@pitrou.net> parents: 55002 diff changeset	184 try:
a0e41dfb972b Add a test for same-thread asynchronous exceptions (see #1779233). Antoine Pitrou <solipsis@pitrou.net> parents: 55002 diff changeset	185 result = set_async_exc(ctypes.c_long(tid), exception)
a0e41dfb972b Add a test for same-thread asynchronous exceptions (see #1779233). Antoine Pitrou <solipsis@pitrou.net> parents: 55002 diff changeset	186 # The exception is async, so we might have to keep the VM busy until
a0e41dfb972b Add a test for same-thread asynchronous exceptions (see #1779233). Antoine Pitrou <solipsis@pitrou.net> parents: 55002 diff changeset	187 # it notices.
a0e41dfb972b Add a test for same-thread asynchronous exceptions (see #1779233). Antoine Pitrou <solipsis@pitrou.net> parents: 55002 diff changeset	188 while True:
a0e41dfb972b Add a test for same-thread asynchronous exceptions (see #1779233). Antoine Pitrou <solipsis@pitrou.net> parents: 55002 diff changeset	189 pass
a0e41dfb972b Add a test for same-thread asynchronous exceptions (see #1779233). Antoine Pitrou <solipsis@pitrou.net> parents: 55002 diff changeset	190 except AsyncExc:
a0e41dfb972b Add a test for same-thread asynchronous exceptions (see #1779233). Antoine Pitrou <solipsis@pitrou.net> parents: 55002 diff changeset	191 pass
a0e41dfb972b Add a test for same-thread asynchronous exceptions (see #1779233). Antoine Pitrou <solipsis@pitrou.net> parents: 55002 diff changeset	192 else:
56600 25ef6f4818a0 Add a comment about unreachable code, and fix a typo Antoine Pitrou <solipsis@pitrou.net> parents: 56598 diff changeset	193 # This code is unreachable but it reflects the intent. If we wanted
25ef6f4818a0 Add a comment about unreachable code, and fix a typo Antoine Pitrou <solipsis@pitrou.net> parents: 56598 diff changeset	194 # to be smarter the above loop wouldn't be infinite.
56598 a0e41dfb972b Add a test for same-thread asynchronous exceptions (see #1779233). Antoine Pitrou <solipsis@pitrou.net> parents: 55002 diff changeset	195 self.fail("AsyncExc not raised")
a0e41dfb972b Add a test for same-thread asynchronous exceptions (see #1779233). Antoine Pitrou <solipsis@pitrou.net> parents: 55002 diff changeset	196 try:
a0e41dfb972b Add a test for same-thread asynchronous exceptions (see #1779233). Antoine Pitrou <solipsis@pitrou.net> parents: 55002 diff changeset	197 self.assertEqual(result, 1) # one thread state modified
a0e41dfb972b Add a test for same-thread asynchronous exceptions (see #1779233). Antoine Pitrou <solipsis@pitrou.net> parents: 55002 diff changeset	198 except UnboundLocalError:
56600 25ef6f4818a0 Add a comment about unreachable code, and fix a typo Antoine Pitrou <solipsis@pitrou.net> parents: 56598 diff changeset	199 # The exception was raised too quickly for us to get the result.
56598 a0e41dfb972b Add a test for same-thread asynchronous exceptions (see #1779233). Antoine Pitrou <solipsis@pitrou.net> parents: 55002 diff changeset	200 pass
a0e41dfb972b Add a test for same-thread asynchronous exceptions (see #1779233). Antoine Pitrou <solipsis@pitrou.net> parents: 55002 diff changeset	201
39431 de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	202 # `worker_started` is set by the thread when it's inside a try/except
de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	203 # block waiting to catch the asynchronously set AsyncExc exception.
de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	204 # `worker_saw_exception` is set by the thread upon catching that
de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	205 # exception.
de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	206 worker_started = threading.Event()
de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	207 worker_saw_exception = threading.Event()
de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	208
de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	209 class Worker(threading.Thread):
de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	210 def run(self):
de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	211 self.id = thread.get_ident()
de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	212 self.finished = False
de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	213
de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	214 try:
de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	215 while True:
de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	216 worker_started.set()
de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	217 time.sleep(0.1)
de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	218 except AsyncExc:
de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	219 self.finished = True
de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	220 worker_saw_exception.set()
de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	221
de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	222 t = Worker()
48288 e143a8f35241 backport threading property changes Benjamin Peterson <benjamin@python.org> parents: 48287 diff changeset	223 t.daemon = True # so if this fails, we don't hang Python at shutdown
39434 be107b04c252 test_PyThreadState_SetAsyncExc(): This is failing on some Tim Peters <tim.peters@gmail.com> parents: 39431 diff changeset	224 t.start()
39431 de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	225 if verbose:
de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	226 print " started worker thread"
de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	227
de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	228 # Try a thread id that doesn't make sense.
de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	229 if verbose:
de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	230 print " trying nonsensical thread id"
39434 be107b04c252 test_PyThreadState_SetAsyncExc(): This is failing on some Tim Peters <tim.peters@gmail.com> parents: 39431 diff changeset	231 result = set_async_exc(ctypes.c_long(-1), exception)
39431 de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	232 self.assertEqual(result, 0) # no thread states modified
de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	233
de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	234 # Now raise an exception in the worker thread.
de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	235 if verbose:
de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	236 print " waiting for worker thread to get started"
52579 b5aa8aa78c0f #1674032: return value of flag from Event.wait(). OKed by Guido. Georg Brandl <georg@python.org> parents: 49361 diff changeset	237 ret = worker_started.wait()
b5aa8aa78c0f #1674032: return value of flag from Event.wait(). OKed by Guido. Georg Brandl <georg@python.org> parents: 49361 diff changeset	238 self.assertTrue(ret)
39431 de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	239 if verbose:
de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	240 print " verifying worker hasn't exited"
55002 6768b2350c57 convert usage of fail* to assert* Benjamin Peterson <benjamin@python.org> parents: 52593 diff changeset	241 self.assertTrue(not t.finished)
39431 de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	242 if verbose:
de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	243 print " attempting to raise asynch exception in worker"
39434 be107b04c252 test_PyThreadState_SetAsyncExc(): This is failing on some Tim Peters <tim.peters@gmail.com> parents: 39431 diff changeset	244 result = set_async_exc(ctypes.c_long(t.id), exception)
39431 de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	245 self.assertEqual(result, 1) # one thread state modified
de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	246 if verbose:
de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	247 print " waiting for worker to say it caught the exception"
de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	248 worker_saw_exception.wait(timeout=10)
55002 6768b2350c57 convert usage of fail* to assert* Benjamin Peterson <benjamin@python.org> parents: 52593 diff changeset	249 self.assertTrue(t.finished)
39431 de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	250 if verbose:
de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	251 print " all OK -- joining worker"
de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	252 if t.finished:
de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	253 t.join()
de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	254 # else the thread is still running, and we have no way to kill it
de9e1a5dca93 Followup to bug #1069160. Tim Peters <tim.peters@gmail.com> parents: 39389 diff changeset	255
59201 2cdec848b74c Merged revisions 78517 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 57159 diff changeset	256 def test_limbo_cleanup(self):
2cdec848b74c Merged revisions 78517 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 57159 diff changeset	257 # Issue 7481: Failure to start thread should cleanup the limbo map.
2cdec848b74c Merged revisions 78517 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 57159 diff changeset	258 def fail_new_thread(*args):
2cdec848b74c Merged revisions 78517 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 57159 diff changeset	259 raise thread.error()
2cdec848b74c Merged revisions 78517 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 57159 diff changeset	260 _start_new_thread = threading._start_new_thread
2cdec848b74c Merged revisions 78517 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 57159 diff changeset	261 threading._start_new_thread = fail_new_thread
2cdec848b74c Merged revisions 78517 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 57159 diff changeset	262 try:
2cdec848b74c Merged revisions 78517 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 57159 diff changeset	263 t = threading.Thread(target=lambda: None)
59220 941e1bf04f55 Merged revisions 78536 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 59201 diff changeset	264 self.assertRaises(thread.error, t.start)
941e1bf04f55 Merged revisions 78536 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 59201 diff changeset	265 self.assertFalse(
941e1bf04f55 Merged revisions 78536 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 59201 diff changeset	266 t in threading._limbo,
941e1bf04f55 Merged revisions 78536 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 59201 diff changeset	267 "Failed to cleanup _limbo map on failure of Thread.start().")
59201 2cdec848b74c Merged revisions 78517 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 57159 diff changeset	268 finally:
2cdec848b74c Merged revisions 78517 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 57159 diff changeset	269 threading._start_new_thread = _start_new_thread
2cdec848b74c Merged revisions 78517 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 57159 diff changeset	270
44063 4417ab5b351e Issue #1402: PyInterpreterState_Clear() may still invoke user code Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 41960 diff changeset	271 def test_finalize_runnning_thread(self):
4417ab5b351e Issue #1402: PyInterpreterState_Clear() may still invoke user code Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 41960 diff changeset	272 # Issue 1402: the PyGILState_Ensure / _Release functions may be called
4417ab5b351e Issue #1402: PyInterpreterState_Clear() may still invoke user code Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 41960 diff changeset	273 # very late on python exit: on deallocation of a running thread for
4417ab5b351e Issue #1402: PyInterpreterState_Clear() may still invoke user code Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 41960 diff changeset	274 # example.
4417ab5b351e Issue #1402: PyInterpreterState_Clear() may still invoke user code Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 41960 diff changeset	275 try:
4417ab5b351e Issue #1402: PyInterpreterState_Clear() may still invoke user code Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 41960 diff changeset	276 import ctypes
4417ab5b351e Issue #1402: PyInterpreterState_Clear() may still invoke user code Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 41960 diff changeset	277 except ImportError:
4417ab5b351e Issue #1402: PyInterpreterState_Clear() may still invoke user code Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 41960 diff changeset	278 if verbose:
4417ab5b351e Issue #1402: PyInterpreterState_Clear() may still invoke user code Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 41960 diff changeset	279 print("test_finalize_with_runnning_thread can't import ctypes")
4417ab5b351e Issue #1402: PyInterpreterState_Clear() may still invoke user code Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 41960 diff changeset	280 return # can't do anything
4417ab5b351e Issue #1402: PyInterpreterState_Clear() may still invoke user code Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 41960 diff changeset	281
4417ab5b351e Issue #1402: PyInterpreterState_Clear() may still invoke user code Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 41960 diff changeset	282 rc = subprocess.call([sys.executable, "-c", """if 1:
4417ab5b351e Issue #1402: PyInterpreterState_Clear() may still invoke user code Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 41960 diff changeset	283 import ctypes, sys, time, thread
4417ab5b351e Issue #1402: PyInterpreterState_Clear() may still invoke user code Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 41960 diff changeset	284
45975 8ef05aceef2b Speed test_threading up from 14s to .5s, and avoid a deadlock on certain Jeffrey Yasskin <jyasskin@gmail.com> parents: 45478 diff changeset	285 # This lock is used as a simple event variable.
8ef05aceef2b Speed test_threading up from 14s to .5s, and avoid a deadlock on certain Jeffrey Yasskin <jyasskin@gmail.com> parents: 45478 diff changeset	286 ready = thread.allocate_lock()
8ef05aceef2b Speed test_threading up from 14s to .5s, and avoid a deadlock on certain Jeffrey Yasskin <jyasskin@gmail.com> parents: 45478 diff changeset	287 ready.acquire()
8ef05aceef2b Speed test_threading up from 14s to .5s, and avoid a deadlock on certain Jeffrey Yasskin <jyasskin@gmail.com> parents: 45478 diff changeset	288
44063 4417ab5b351e Issue #1402: PyInterpreterState_Clear() may still invoke user code Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 41960 diff changeset	289 # Module globals are cleared before __del__ is run
4417ab5b351e Issue #1402: PyInterpreterState_Clear() may still invoke user code Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 41960 diff changeset	290 # So we save the functions in class dict
4417ab5b351e Issue #1402: PyInterpreterState_Clear() may still invoke user code Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 41960 diff changeset	291 class C:
4417ab5b351e Issue #1402: PyInterpreterState_Clear() may still invoke user code Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 41960 diff changeset	292 ensure = ctypes.pythonapi.PyGILState_Ensure
4417ab5b351e Issue #1402: PyInterpreterState_Clear() may still invoke user code Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 41960 diff changeset	293 release = ctypes.pythonapi.PyGILState_Release
4417ab5b351e Issue #1402: PyInterpreterState_Clear() may still invoke user code Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 41960 diff changeset	294 def __del__(self):
4417ab5b351e Issue #1402: PyInterpreterState_Clear() may still invoke user code Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 41960 diff changeset	295 state = self.ensure()
4417ab5b351e Issue #1402: PyInterpreterState_Clear() may still invoke user code Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 41960 diff changeset	296 self.release(state)
4417ab5b351e Issue #1402: PyInterpreterState_Clear() may still invoke user code Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 41960 diff changeset	297
4417ab5b351e Issue #1402: PyInterpreterState_Clear() may still invoke user code Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 41960 diff changeset	298 def waitingThread():
4417ab5b351e Issue #1402: PyInterpreterState_Clear() may still invoke user code Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 41960 diff changeset	299 x = C()
45975 8ef05aceef2b Speed test_threading up from 14s to .5s, and avoid a deadlock on certain Jeffrey Yasskin <jyasskin@gmail.com> parents: 45478 diff changeset	300 ready.release()
44063 4417ab5b351e Issue #1402: PyInterpreterState_Clear() may still invoke user code Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 41960 diff changeset	301 time.sleep(100)
4417ab5b351e Issue #1402: PyInterpreterState_Clear() may still invoke user code Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 41960 diff changeset	302
4417ab5b351e Issue #1402: PyInterpreterState_Clear() may still invoke user code Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 41960 diff changeset	303 thread.start_new_thread(waitingThread, ())
45975 8ef05aceef2b Speed test_threading up from 14s to .5s, and avoid a deadlock on certain Jeffrey Yasskin <jyasskin@gmail.com> parents: 45478 diff changeset	304 ready.acquire() # Be sure the other thread is waiting.
44063 4417ab5b351e Issue #1402: PyInterpreterState_Clear() may still invoke user code Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 41960 diff changeset	305 sys.exit(42)
4417ab5b351e Issue #1402: PyInterpreterState_Clear() may still invoke user code Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 41960 diff changeset	306 """])
4417ab5b351e Issue #1402: PyInterpreterState_Clear() may still invoke user code Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 41960 diff changeset	307 self.assertEqual(rc, 42)
4417ab5b351e Issue #1402: PyInterpreterState_Clear() may still invoke user code Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 41960 diff changeset	308
46280 76f577a9ec03 #1733757: the interpreter would hang on shutdown, if the function set by sys.settrace Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 46146 diff changeset	309 def test_finalize_with_trace(self):
76f577a9ec03 #1733757: the interpreter would hang on shutdown, if the function set by sys.settrace Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 46146 diff changeset	310 # Issue1733757
76f577a9ec03 #1733757: the interpreter would hang on shutdown, if the function set by sys.settrace Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 46146 diff changeset	311 # Avoid a deadlock when sys.settrace steps into threading._shutdown
64854 6bd276f5d1bc Merged revisions 84909-84913 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 61046 diff changeset	312 p = subprocess.Popen([sys.executable, "-c", """if 1:
46280 76f577a9ec03 #1733757: the interpreter would hang on shutdown, if the function set by sys.settrace Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 46146 diff changeset	313 import sys, threading
76f577a9ec03 #1733757: the interpreter would hang on shutdown, if the function set by sys.settrace Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 46146 diff changeset	314
76f577a9ec03 #1733757: the interpreter would hang on shutdown, if the function set by sys.settrace Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 46146 diff changeset	315 # A deadlock-killer, to prevent the
76f577a9ec03 #1733757: the interpreter would hang on shutdown, if the function set by sys.settrace Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 46146 diff changeset	316 # testsuite to hang forever
76f577a9ec03 #1733757: the interpreter would hang on shutdown, if the function set by sys.settrace Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 46146 diff changeset	317 def killer():
76f577a9ec03 #1733757: the interpreter would hang on shutdown, if the function set by sys.settrace Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 46146 diff changeset	318 import os, time
76f577a9ec03 #1733757: the interpreter would hang on shutdown, if the function set by sys.settrace Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 46146 diff changeset	319 time.sleep(2)
76f577a9ec03 #1733757: the interpreter would hang on shutdown, if the function set by sys.settrace Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 46146 diff changeset	320 print 'program blocked; aborting'
76f577a9ec03 #1733757: the interpreter would hang on shutdown, if the function set by sys.settrace Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 46146 diff changeset	321 os._exit(2)
76f577a9ec03 #1733757: the interpreter would hang on shutdown, if the function set by sys.settrace Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 46146 diff changeset	322 t = threading.Thread(target=killer)
48288 e143a8f35241 backport threading property changes Benjamin Peterson <benjamin@python.org> parents: 48287 diff changeset	323 t.daemon = True
46280 76f577a9ec03 #1733757: the interpreter would hang on shutdown, if the function set by sys.settrace Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 46146 diff changeset	324 t.start()
76f577a9ec03 #1733757: the interpreter would hang on shutdown, if the function set by sys.settrace Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 46146 diff changeset	325
76f577a9ec03 #1733757: the interpreter would hang on shutdown, if the function set by sys.settrace Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 46146 diff changeset	326 # This is the trace function
76f577a9ec03 #1733757: the interpreter would hang on shutdown, if the function set by sys.settrace Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 46146 diff changeset	327 def func(frame, event, arg):
47598 c349f9fbc704 give the threading API PEP 8 names Benjamin Peterson <benjamin@python.org> parents: 47438 diff changeset	328 threading.current_thread()
46280 76f577a9ec03 #1733757: the interpreter would hang on shutdown, if the function set by sys.settrace Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 46146 diff changeset	329 return func
76f577a9ec03 #1733757: the interpreter would hang on shutdown, if the function set by sys.settrace Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 46146 diff changeset	330
76f577a9ec03 #1733757: the interpreter would hang on shutdown, if the function set by sys.settrace Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 46146 diff changeset	331 sys.settrace(func)
64854 6bd276f5d1bc Merged revisions 84909-84913 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 61046 diff changeset	332 """],
6bd276f5d1bc Merged revisions 84909-84913 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 61046 diff changeset	333 stdout=subprocess.PIPE,
6bd276f5d1bc Merged revisions 84909-84913 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 61046 diff changeset	334 stderr=subprocess.PIPE)
65982 c37bef8dcea2 Merged revisions 86209 via svnmerge from Brian Curtin <brian.curtin@gmail.com> parents: 65886 diff changeset	335 self.addCleanup(p.stdout.close)
c37bef8dcea2 Merged revisions 86209 via svnmerge from Brian Curtin <brian.curtin@gmail.com> parents: 65886 diff changeset	336 self.addCleanup(p.stderr.close)
64854 6bd276f5d1bc Merged revisions 84909-84913 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 61046 diff changeset	337 stdout, stderr = p.communicate()
6bd276f5d1bc Merged revisions 84909-84913 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 61046 diff changeset	338 rc = p.returncode
55002 6768b2350c57 convert usage of fail* to assert* Benjamin Peterson <benjamin@python.org> parents: 52593 diff changeset	339 self.assertFalse(rc == 2, "interpreted was blocked")
64854 6bd276f5d1bc Merged revisions 84909-84913 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 61046 diff changeset	340 self.assertTrue(rc == 0,
6bd276f5d1bc Merged revisions 84909-84913 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 61046 diff changeset	341 "Unexpected error: " + repr(stderr))
46280 76f577a9ec03 #1733757: the interpreter would hang on shutdown, if the function set by sys.settrace Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 46146 diff changeset	342
56781 2abc3b4bce7c Merged revisions 75570,75574,75624 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52596 diff changeset	343 def test_join_nondaemon_on_shutdown(self):
2abc3b4bce7c Merged revisions 75570,75574,75624 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52596 diff changeset	344 # Issue 1722344
2abc3b4bce7c Merged revisions 75570,75574,75624 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52596 diff changeset	345 # Raising SystemExit skipped threading._shutdown
2abc3b4bce7c Merged revisions 75570,75574,75624 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52596 diff changeset	346 p = subprocess.Popen([sys.executable, "-c", """if 1:
2abc3b4bce7c Merged revisions 75570,75574,75624 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52596 diff changeset	347 import threading
2abc3b4bce7c Merged revisions 75570,75574,75624 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52596 diff changeset	348 from time import sleep
2abc3b4bce7c Merged revisions 75570,75574,75624 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52596 diff changeset	349
2abc3b4bce7c Merged revisions 75570,75574,75624 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52596 diff changeset	350 def child():
2abc3b4bce7c Merged revisions 75570,75574,75624 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52596 diff changeset	351 sleep(1)
2abc3b4bce7c Merged revisions 75570,75574,75624 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52596 diff changeset	352 # As a non-daemon thread we SHOULD wake up and nothing
2abc3b4bce7c Merged revisions 75570,75574,75624 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52596 diff changeset	353 # should be torn down yet
2abc3b4bce7c Merged revisions 75570,75574,75624 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52596 diff changeset	354 print "Woke up, sleep function is:", sleep
2abc3b4bce7c Merged revisions 75570,75574,75624 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52596 diff changeset	355
2abc3b4bce7c Merged revisions 75570,75574,75624 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52596 diff changeset	356 threading.Thread(target=child).start()
2abc3b4bce7c Merged revisions 75570,75574,75624 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52596 diff changeset	357 raise SystemExit
2abc3b4bce7c Merged revisions 75570,75574,75624 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52596 diff changeset	358 """],
2abc3b4bce7c Merged revisions 75570,75574,75624 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52596 diff changeset	359 stdout=subprocess.PIPE,
2abc3b4bce7c Merged revisions 75570,75574,75624 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52596 diff changeset	360 stderr=subprocess.PIPE)
65982 c37bef8dcea2 Merged revisions 86209 via svnmerge from Brian Curtin <brian.curtin@gmail.com> parents: 65886 diff changeset	361 self.addCleanup(p.stdout.close)
c37bef8dcea2 Merged revisions 86209 via svnmerge from Brian Curtin <brian.curtin@gmail.com> parents: 65886 diff changeset	362 self.addCleanup(p.stderr.close)
56781 2abc3b4bce7c Merged revisions 75570,75574,75624 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52596 diff changeset	363 stdout, stderr = p.communicate()
2abc3b4bce7c Merged revisions 75570,75574,75624 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52596 diff changeset	364 self.assertEqual(stdout.strip(),
2abc3b4bce7c Merged revisions 75570,75574,75624 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52596 diff changeset	365 "Woke up, sleep function is: <built-in function sleep>")
2abc3b4bce7c Merged revisions 75570,75574,75624 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52596 diff changeset	366 stderr = re.sub(r"^\[\d+ refs\]", "", stderr, re.MULTILINE).strip()
2abc3b4bce7c Merged revisions 75570,75574,75624 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52596 diff changeset	367 self.assertEqual(stderr, "")
46280 76f577a9ec03 #1733757: the interpreter would hang on shutdown, if the function set by sys.settrace Amaury Forgeot d'Arc <amauryfa@gmail.com> parents: 46146 diff changeset	368
44871 ffacb44673ea Backport of r60190: Gregory P. Smith <greg@mad-scientist.com> parents: 39434 diff changeset	369 def test_enumerate_after_join(self):
ffacb44673ea Backport of r60190: Gregory P. Smith <greg@mad-scientist.com> parents: 39434 diff changeset	370 # Try hard to trigger #1703448: a thread is still returned in
ffacb44673ea Backport of r60190: Gregory P. Smith <greg@mad-scientist.com> parents: 39434 diff changeset	371 # threading.enumerate() after it has been join()ed.
ffacb44673ea Backport of r60190: Gregory P. Smith <greg@mad-scientist.com> parents: 39434 diff changeset	372 enum = threading.enumerate
ffacb44673ea Backport of r60190: Gregory P. Smith <greg@mad-scientist.com> parents: 39434 diff changeset	373 old_interval = sys.getcheckinterval()
ffacb44673ea Backport of r60190: Gregory P. Smith <greg@mad-scientist.com> parents: 39434 diff changeset	374 try:
45975 8ef05aceef2b Speed test_threading up from 14s to .5s, and avoid a deadlock on certain Jeffrey Yasskin <jyasskin@gmail.com> parents: 45478 diff changeset	375 for i in xrange(1, 100):
8ef05aceef2b Speed test_threading up from 14s to .5s, and avoid a deadlock on certain Jeffrey Yasskin <jyasskin@gmail.com> parents: 45478 diff changeset	376 # Try a couple times at each thread-switching interval
8ef05aceef2b Speed test_threading up from 14s to .5s, and avoid a deadlock on certain Jeffrey Yasskin <jyasskin@gmail.com> parents: 45478 diff changeset	377 # to get more interleavings.
8ef05aceef2b Speed test_threading up from 14s to .5s, and avoid a deadlock on certain Jeffrey Yasskin <jyasskin@gmail.com> parents: 45478 diff changeset	378 sys.setcheckinterval(i // 5)
44871 ffacb44673ea Backport of r60190: Gregory P. Smith <greg@mad-scientist.com> parents: 39434 diff changeset	379 t = threading.Thread(target=lambda: None)
ffacb44673ea Backport of r60190: Gregory P. Smith <greg@mad-scientist.com> parents: 39434 diff changeset	380 t.start()
ffacb44673ea Backport of r60190: Gregory P. Smith <greg@mad-scientist.com> parents: 39434 diff changeset	381 t.join()
ffacb44673ea Backport of r60190: Gregory P. Smith <greg@mad-scientist.com> parents: 39434 diff changeset	382 l = enum()
58460 17891566a478 use assert[Not]In where appropriate Ezio Melotti <ezio.melotti@gmail.com> parents: 57157 diff changeset	383 self.assertNotIn(t, l,
44871 ffacb44673ea Backport of r60190: Gregory P. Smith <greg@mad-scientist.com> parents: 39434 diff changeset	384 "#1703448 triggered after %d trials: %s" % (i, l))
ffacb44673ea Backport of r60190: Gregory P. Smith <greg@mad-scientist.com> parents: 39434 diff changeset	385 finally:
ffacb44673ea Backport of r60190: Gregory P. Smith <greg@mad-scientist.com> parents: 39434 diff changeset	386 sys.setcheckinterval(old_interval)
ffacb44673ea Backport of r60190: Gregory P. Smith <greg@mad-scientist.com> parents: 39434 diff changeset	387
45477 241cf81a0508 Prevent classes like: Jeffrey Yasskin <jyasskin@gmail.com> parents: 44870 diff changeset	388 def test_no_refcycle_through_target(self):
241cf81a0508 Prevent classes like: Jeffrey Yasskin <jyasskin@gmail.com> parents: 44870 diff changeset	389 class RunSelfFunction(object):
45478 9f183ae3458b Followup to r61011: Also avoid the reference cycle when the Thread's target Jeffrey Yasskin <jyasskin@gmail.com> parents: 45477 diff changeset	390 def __init__(self, should_raise):
45477 241cf81a0508 Prevent classes like: Jeffrey Yasskin <jyasskin@gmail.com> parents: 44870 diff changeset	391 # The links in this refcycle from Thread back to self
241cf81a0508 Prevent classes like: Jeffrey Yasskin <jyasskin@gmail.com> parents: 44870 diff changeset	392 # should be cleaned up when the thread completes.
45478 9f183ae3458b Followup to r61011: Also avoid the reference cycle when the Thread's target Jeffrey Yasskin <jyasskin@gmail.com> parents: 45477 diff changeset	393 self.should_raise = should_raise
45477 241cf81a0508 Prevent classes like: Jeffrey Yasskin <jyasskin@gmail.com> parents: 44870 diff changeset	394 self.thread = threading.Thread(target=self._run,
241cf81a0508 Prevent classes like: Jeffrey Yasskin <jyasskin@gmail.com> parents: 44870 diff changeset	395 args=(self,),
241cf81a0508 Prevent classes like: Jeffrey Yasskin <jyasskin@gmail.com> parents: 44870 diff changeset	396 kwargs={'yet_another':self})
241cf81a0508 Prevent classes like: Jeffrey Yasskin <jyasskin@gmail.com> parents: 44870 diff changeset	397 self.thread.start()
241cf81a0508 Prevent classes like: Jeffrey Yasskin <jyasskin@gmail.com> parents: 44870 diff changeset	398
241cf81a0508 Prevent classes like: Jeffrey Yasskin <jyasskin@gmail.com> parents: 44870 diff changeset	399 def _run(self, other_ref, yet_another):
45478 9f183ae3458b Followup to r61011: Also avoid the reference cycle when the Thread's target Jeffrey Yasskin <jyasskin@gmail.com> parents: 45477 diff changeset	400 if self.should_raise:
9f183ae3458b Followup to r61011: Also avoid the reference cycle when the Thread's target Jeffrey Yasskin <jyasskin@gmail.com> parents: 45477 diff changeset	401 raise SystemExit
45477 241cf81a0508 Prevent classes like: Jeffrey Yasskin <jyasskin@gmail.com> parents: 44870 diff changeset	402
45478 9f183ae3458b Followup to r61011: Also avoid the reference cycle when the Thread's target Jeffrey Yasskin <jyasskin@gmail.com> parents: 45477 diff changeset	403 cyclic_object = RunSelfFunction(should_raise=False)
45477 241cf81a0508 Prevent classes like: Jeffrey Yasskin <jyasskin@gmail.com> parents: 44870 diff changeset	404 weak_cyclic_object = weakref.ref(cyclic_object)
241cf81a0508 Prevent classes like: Jeffrey Yasskin <jyasskin@gmail.com> parents: 44870 diff changeset	405 cyclic_object.thread.join()
241cf81a0508 Prevent classes like: Jeffrey Yasskin <jyasskin@gmail.com> parents: 44870 diff changeset	406 del cyclic_object
66348 614c84670c63 Merged revisions 86596 via svnmerge from Ezio Melotti <ezio.melotti@gmail.com> parents: 65982 diff changeset	407 self.assertEqual(None, weak_cyclic_object(),
614c84670c63 Merged revisions 86596 via svnmerge from Ezio Melotti <ezio.melotti@gmail.com> parents: 65982 diff changeset	408 msg=('%d references still around' %
614c84670c63 Merged revisions 86596 via svnmerge from Ezio Melotti <ezio.melotti@gmail.com> parents: 65982 diff changeset	409 sys.getrefcount(weak_cyclic_object())))
45477 241cf81a0508 Prevent classes like: Jeffrey Yasskin <jyasskin@gmail.com> parents: 44870 diff changeset	410
45478 9f183ae3458b Followup to r61011: Also avoid the reference cycle when the Thread's target Jeffrey Yasskin <jyasskin@gmail.com> parents: 45477 diff changeset	411 raising_cyclic_object = RunSelfFunction(should_raise=True)
9f183ae3458b Followup to r61011: Also avoid the reference cycle when the Thread's target Jeffrey Yasskin <jyasskin@gmail.com> parents: 45477 diff changeset	412 weak_raising_cyclic_object = weakref.ref(raising_cyclic_object)
9f183ae3458b Followup to r61011: Also avoid the reference cycle when the Thread's target Jeffrey Yasskin <jyasskin@gmail.com> parents: 45477 diff changeset	413 raising_cyclic_object.thread.join()
9f183ae3458b Followup to r61011: Also avoid the reference cycle when the Thread's target Jeffrey Yasskin <jyasskin@gmail.com> parents: 45477 diff changeset	414 del raising_cyclic_object
66348 614c84670c63 Merged revisions 86596 via svnmerge from Ezio Melotti <ezio.melotti@gmail.com> parents: 65982 diff changeset	415 self.assertEqual(None, weak_raising_cyclic_object(),
614c84670c63 Merged revisions 86596 via svnmerge from Ezio Melotti <ezio.melotti@gmail.com> parents: 65982 diff changeset	416 msg=('%d references still around' %
614c84670c63 Merged revisions 86596 via svnmerge from Ezio Melotti <ezio.melotti@gmail.com> parents: 65982 diff changeset	417 sys.getrefcount(weak_raising_cyclic_object())))
45478 9f183ae3458b Followup to r61011: Also avoid the reference cycle when the Thread's target Jeffrey Yasskin <jyasskin@gmail.com> parents: 45477 diff changeset	418
76420 ab9d6c4907e7 Issue #14308: Fix an exception when a "dummy" thread is in the threading module's active list after a fork(). Antoine Pitrou <solipsis@pitrou.net> parents: 74829 diff changeset	419 @unittest.skipUnless(hasattr(os, 'fork'), 'test needs fork()')
ab9d6c4907e7 Issue #14308: Fix an exception when a "dummy" thread is in the threading module's active list after a fork(). Antoine Pitrou <solipsis@pitrou.net> parents: 74829 diff changeset	420 def test_dummy_thread_after_fork(self):
ab9d6c4907e7 Issue #14308: Fix an exception when a "dummy" thread is in the threading module's active list after a fork(). Antoine Pitrou <solipsis@pitrou.net> parents: 74829 diff changeset	421 # Issue #14308: a dummy thread in the active list doesn't mess up
ab9d6c4907e7 Issue #14308: Fix an exception when a "dummy" thread is in the threading module's active list after a fork(). Antoine Pitrou <solipsis@pitrou.net> parents: 74829 diff changeset	422 # the after-fork mechanism.
ab9d6c4907e7 Issue #14308: Fix an exception when a "dummy" thread is in the threading module's active list after a fork(). Antoine Pitrou <solipsis@pitrou.net> parents: 74829 diff changeset	423 code = """if 1:
ab9d6c4907e7 Issue #14308: Fix an exception when a "dummy" thread is in the threading module's active list after a fork(). Antoine Pitrou <solipsis@pitrou.net> parents: 74829 diff changeset	424 import thread, threading, os, time
ab9d6c4907e7 Issue #14308: Fix an exception when a "dummy" thread is in the threading module's active list after a fork(). Antoine Pitrou <solipsis@pitrou.net> parents: 74829 diff changeset	425
ab9d6c4907e7 Issue #14308: Fix an exception when a "dummy" thread is in the threading module's active list after a fork(). Antoine Pitrou <solipsis@pitrou.net> parents: 74829 diff changeset	426 def background_thread(evt):
ab9d6c4907e7 Issue #14308: Fix an exception when a "dummy" thread is in the threading module's active list after a fork(). Antoine Pitrou <solipsis@pitrou.net> parents: 74829 diff changeset	427 # Creates and registers the _DummyThread instance
ab9d6c4907e7 Issue #14308: Fix an exception when a "dummy" thread is in the threading module's active list after a fork(). Antoine Pitrou <solipsis@pitrou.net> parents: 74829 diff changeset	428 threading.current_thread()
ab9d6c4907e7 Issue #14308: Fix an exception when a "dummy" thread is in the threading module's active list after a fork(). Antoine Pitrou <solipsis@pitrou.net> parents: 74829 diff changeset	429 evt.set()
ab9d6c4907e7 Issue #14308: Fix an exception when a "dummy" thread is in the threading module's active list after a fork(). Antoine Pitrou <solipsis@pitrou.net> parents: 74829 diff changeset	430 time.sleep(10)
ab9d6c4907e7 Issue #14308: Fix an exception when a "dummy" thread is in the threading module's active list after a fork(). Antoine Pitrou <solipsis@pitrou.net> parents: 74829 diff changeset	431
ab9d6c4907e7 Issue #14308: Fix an exception when a "dummy" thread is in the threading module's active list after a fork(). Antoine Pitrou <solipsis@pitrou.net> parents: 74829 diff changeset	432 evt = threading.Event()
ab9d6c4907e7 Issue #14308: Fix an exception when a "dummy" thread is in the threading module's active list after a fork(). Antoine Pitrou <solipsis@pitrou.net> parents: 74829 diff changeset	433 thread.start_new_thread(background_thread, (evt,))
ab9d6c4907e7 Issue #14308: Fix an exception when a "dummy" thread is in the threading module's active list after a fork(). Antoine Pitrou <solipsis@pitrou.net> parents: 74829 diff changeset	434 evt.wait()
ab9d6c4907e7 Issue #14308: Fix an exception when a "dummy" thread is in the threading module's active list after a fork(). Antoine Pitrou <solipsis@pitrou.net> parents: 74829 diff changeset	435 assert threading.active_count() == 2, threading.active_count()
ab9d6c4907e7 Issue #14308: Fix an exception when a "dummy" thread is in the threading module's active list after a fork(). Antoine Pitrou <solipsis@pitrou.net> parents: 74829 diff changeset	436 if os.fork() == 0:
ab9d6c4907e7 Issue #14308: Fix an exception when a "dummy" thread is in the threading module's active list after a fork(). Antoine Pitrou <solipsis@pitrou.net> parents: 74829 diff changeset	437 assert threading.active_count() == 1, threading.active_count()
ab9d6c4907e7 Issue #14308: Fix an exception when a "dummy" thread is in the threading module's active list after a fork(). Antoine Pitrou <solipsis@pitrou.net> parents: 74829 diff changeset	438 os._exit(0)
ab9d6c4907e7 Issue #14308: Fix an exception when a "dummy" thread is in the threading module's active list after a fork(). Antoine Pitrou <solipsis@pitrou.net> parents: 74829 diff changeset	439 else:
ab9d6c4907e7 Issue #14308: Fix an exception when a "dummy" thread is in the threading module's active list after a fork(). Antoine Pitrou <solipsis@pitrou.net> parents: 74829 diff changeset	440 os.wait()
ab9d6c4907e7 Issue #14308: Fix an exception when a "dummy" thread is in the threading module's active list after a fork(). Antoine Pitrou <solipsis@pitrou.net> parents: 74829 diff changeset	441 """
ab9d6c4907e7 Issue #14308: Fix an exception when a "dummy" thread is in the threading module's active list after a fork(). Antoine Pitrou <solipsis@pitrou.net> parents: 74829 diff changeset	442 _, out, err = assert_python_ok("-c", code)
ab9d6c4907e7 Issue #14308: Fix an exception when a "dummy" thread is in the threading module's active list after a fork(). Antoine Pitrou <solipsis@pitrou.net> parents: 74829 diff changeset	443 self.assertEqual(out, '')
ab9d6c4907e7 Issue #14308: Fix an exception when a "dummy" thread is in the threading module's active list after a fork(). Antoine Pitrou <solipsis@pitrou.net> parents: 74829 diff changeset	444 self.assertEqual(err, '')
ab9d6c4907e7 Issue #14308: Fix an exception when a "dummy" thread is in the threading module's active list after a fork(). Antoine Pitrou <solipsis@pitrou.net> parents: 74829 diff changeset	445
44871 ffacb44673ea Backport of r60190: Gregory P. Smith <greg@mad-scientist.com> parents: 39434 diff changeset	446
56882 5f61bd3aa427 Suppress transient refleaks in test_threading. Antoine Pitrou <solipsis@pitrou.net> parents: 56686 diff changeset	447 class ThreadJoinOnShutdown(BaseTestCase):
48274 43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	448
71114 ff36b8cadfd6 Issue #11870: Skip test_threading.test_2_join_in_forked_process() on platforms Victor Stinner <victor.stinner@haypocalc.com> parents: 70447 diff changeset	449 # Between fork() and exec(), only async-safe functions are allowed (issues
ff36b8cadfd6 Issue #11870: Skip test_threading.test_2_join_in_forked_process() on platforms Victor Stinner <victor.stinner@haypocalc.com> parents: 70447 diff changeset	450 # #12316 and #11870), and fork() from a worker thread is known to trigger
ff36b8cadfd6 Issue #11870: Skip test_threading.test_2_join_in_forked_process() on platforms Victor Stinner <victor.stinner@haypocalc.com> parents: 70447 diff changeset	451 # problems with some operating systems (issue #3863): skip problematic tests
ff36b8cadfd6 Issue #11870: Skip test_threading.test_2_join_in_forked_process() on platforms Victor Stinner <victor.stinner@haypocalc.com> parents: 70447 diff changeset	452 # on platforms known to behave badly.
ff36b8cadfd6 Issue #11870: Skip test_threading.test_2_join_in_forked_process() on platforms Victor Stinner <victor.stinner@haypocalc.com> parents: 70447 diff changeset	453 platforms_to_skip = ('freebsd4', 'freebsd5', 'freebsd6', 'netbsd5',
ff36b8cadfd6 Issue #11870: Skip test_threading.test_2_join_in_forked_process() on platforms Victor Stinner <victor.stinner@haypocalc.com> parents: 70447 diff changeset	454 'os2emx')
ff36b8cadfd6 Issue #11870: Skip test_threading.test_2_join_in_forked_process() on platforms Victor Stinner <victor.stinner@haypocalc.com> parents: 70447 diff changeset	455
48274 43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	456 def _run_and_join(self, script):
43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	457 script = """if 1:
43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	458 import sys, os, time, threading
43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	459
43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	460 # a thread, which waits for the main program to terminate
43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	461 def joiningfunc(mainthread):
43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	462 mainthread.join()
43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	463 print 'end of thread'
43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	464 \n""" + script
43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	465
43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	466 p = subprocess.Popen([sys.executable, "-c", script], stdout=subprocess.PIPE)
43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	467 rc = p.wait()
48844 85c08cf08b2b let Windows bots pass on test_threading Benjamin Peterson <benjamin@python.org> parents: 48274 diff changeset	468 data = p.stdout.read().replace('\r', '')
65886 deb23cbab5ab Merged revisions 86107 via svnmerge from Brian Curtin <brian.curtin@gmail.com> parents: 65430 diff changeset	469 p.stdout.close()
48844 85c08cf08b2b let Windows bots pass on test_threading Benjamin Peterson <benjamin@python.org> parents: 48274 diff changeset	470 self.assertEqual(data, "end of main\nend of thread\n")
55002 6768b2350c57 convert usage of fail* to assert* Benjamin Peterson <benjamin@python.org> parents: 52593 diff changeset	471 self.assertFalse(rc == 2, "interpreter was blocked")
6768b2350c57 convert usage of fail* to assert* Benjamin Peterson <benjamin@python.org> parents: 52593 diff changeset	472 self.assertTrue(rc == 0, "Unexpected error")
48274 43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	473
43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	474 def test_1_join_on_shutdown(self):
43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	475 # The usual case: on exit, wait for a non-daemon thread
43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	476 script = """if 1:
43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	477 import os
43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	478 t = threading.Thread(target=joiningfunc,
47985 09beb96ba20d Apply patch for 874900: threading module can deadlock after fork Jesse Noller <jnoller@gmail.com> parents: 47598 diff changeset	479 args=(threading.current_thread(),))
48274 43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	480 t.start()
43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	481 time.sleep(0.1)
43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	482 print 'end of main'
43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	483 """
43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	484 self._run_and_join(script)
43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	485
43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	486
71114 ff36b8cadfd6 Issue #11870: Skip test_threading.test_2_join_in_forked_process() on platforms Victor Stinner <victor.stinner@haypocalc.com> parents: 70447 diff changeset	487 @unittest.skipUnless(hasattr(os, 'fork'), "needs os.fork()")
ff36b8cadfd6 Issue #11870: Skip test_threading.test_2_join_in_forked_process() on platforms Victor Stinner <victor.stinner@haypocalc.com> parents: 70447 diff changeset	488 @unittest.skipIf(sys.platform in platforms_to_skip, "due to known OS bug")
48274 43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	489 def test_2_join_in_forked_process(self):
43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	490 # Like the test above, but from a forked interpreter
43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	491 script = """if 1:
43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	492 childpid = os.fork()
43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	493 if childpid != 0:
43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	494 os.waitpid(childpid, 0)
43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	495 sys.exit(0)
43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	496
43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	497 t = threading.Thread(target=joiningfunc,
47985 09beb96ba20d Apply patch for 874900: threading module can deadlock after fork Jesse Noller <jnoller@gmail.com> parents: 47598 diff changeset	498 args=(threading.current_thread(),))
48274 43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	499 t.start()
43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	500 print 'end of main'
43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	501 """
43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	502 self._run_and_join(script)
43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	503
71114 ff36b8cadfd6 Issue #11870: Skip test_threading.test_2_join_in_forked_process() on platforms Victor Stinner <victor.stinner@haypocalc.com> parents: 70447 diff changeset	504 @unittest.skipUnless(hasattr(os, 'fork'), "needs os.fork()")
ff36b8cadfd6 Issue #11870: Skip test_threading.test_2_join_in_forked_process() on platforms Victor Stinner <victor.stinner@haypocalc.com> parents: 70447 diff changeset	505 @unittest.skipIf(sys.platform in platforms_to_skip, "due to known OS bug")
48274 43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	506 def test_3_join_in_forked_from_thread(self):
43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	507 # Like the test above, but fork() was called from a worker thread
43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	508 # In the forked process, the main Thread object must be marked as stopped.
43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	509 script = """if 1:
47985 09beb96ba20d Apply patch for 874900: threading module can deadlock after fork Jesse Noller <jnoller@gmail.com> parents: 47598 diff changeset	510 main_thread = threading.current_thread()
48274 43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	511 def worker():
43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	512 childpid = os.fork()
43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	513 if childpid != 0:
43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	514 os.waitpid(childpid, 0)
43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	515 sys.exit(0)
43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	516
43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	517 t = threading.Thread(target=joiningfunc,
43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	518 args=(main_thread,))
43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	519 print 'end of main'
43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	520 t.start()
43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	521 t.join() # Should not block: main_thread is already stopped
43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	522
43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	523 w = threading.Thread(target=worker)
43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	524 w.start()
43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	525 """
43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	526 self._run_and_join(script)
43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	527
67341 3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	528 def assertScriptHasOutput(self, script, expected_output):
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	529 p = subprocess.Popen([sys.executable, "-c", script],
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	530 stdout=subprocess.PIPE)
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	531 rc = p.wait()
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	532 data = p.stdout.read().decode().replace('\r', '')
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	533 self.assertEqual(rc, 0, "Unexpected error")
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	534 self.assertEqual(data, expected_output)
48274 43fa50b9eb12 Backport of r65032 from trunk Gregory P. Smith <greg@mad-scientist.com> parents: 44871 diff changeset	535
67341 3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	536 @unittest.skipUnless(hasattr(os, 'fork'), "needs os.fork()")
71114 ff36b8cadfd6 Issue #11870: Skip test_threading.test_2_join_in_forked_process() on platforms Victor Stinner <victor.stinner@haypocalc.com> parents: 70447 diff changeset	537 @unittest.skipIf(sys.platform in platforms_to_skip, "due to known OS bug")
67341 3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	538 def test_4_joining_across_fork_in_worker_thread(self):
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	539 # There used to be a possible deadlock when forking from a child
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	540 # thread. See http://bugs.python.org/issue6643.
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	541
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	542 # The script takes the following steps:
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	543 # - The main thread in the parent process starts a new thread and then
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	544 # tries to join it.
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	545 # - The join operation acquires the Lock inside the thread's _block
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	546 # Condition. (See threading.py:Thread.join().)
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	547 # - We stub out the acquire method on the condition to force it to wait
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	548 # until the child thread forks. (See LOCK ACQUIRED HERE)
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	549 # - The child thread forks. (See LOCK HELD and WORKER THREAD FORKS
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	550 # HERE)
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	551 # - The main thread of the parent process enters Condition.wait(),
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	552 # which releases the lock on the child thread.
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	553 # - The child process returns. Without the necessary fix, when the
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	554 # main thread of the child process (which used to be the child thread
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	555 # in the parent process) attempts to exit, it will try to acquire the
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	556 # lock in the Thread._block Condition object and hang, because the
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	557 # lock was held across the fork.
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	558
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	559 script = """if 1:
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	560 import os, time, threading
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	561
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	562 finish_join = False
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	563 start_fork = False
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	564
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	565 def worker():
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	566 # Wait until this thread's lock is acquired before forking to
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	567 # create the deadlock.
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	568 global finish_join
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	569 while not start_fork:
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	570 time.sleep(0.01)
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	571 # LOCK HELD: Main thread holds lock across this call.
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	572 childpid = os.fork()
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	573 finish_join = True
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	574 if childpid != 0:
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	575 # Parent process just waits for child.
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	576 os.waitpid(childpid, 0)
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	577 # Child process should just return.
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	578
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	579 w = threading.Thread(target=worker)
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	580
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	581 # Stub out the private condition variable's lock acquire method.
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	582 # This acquires the lock and then waits until the child has forked
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	583 # before returning, which will release the lock soon after. If
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	584 # someone else tries to fix this test case by acquiring this lock
68588 cd8d3c2d3bce #11565: Fix several typos. Patch by Piotr Kasprzyk. Ezio Melotti <ezio.melotti@gmail.com> parents: 68256 diff changeset	585 # before forking instead of resetting it, the test case will
67341 3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	586 # deadlock when it shouldn't.
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	587 condition = w._block
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	588 orig_acquire = condition.acquire
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	589 call_count_lock = threading.Lock()
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	590 call_count = 0
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	591 def my_acquire():
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	592 global call_count
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	593 global start_fork
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	594 orig_acquire() # LOCK ACQUIRED HERE
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	595 start_fork = True
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	596 if call_count == 0:
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	597 while not finish_join:
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	598 time.sleep(0.01) # WORKER THREAD FORKS HERE
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	599 with call_count_lock:
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	600 call_count += 1
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	601 condition.acquire = my_acquire
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	602
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	603 w.start()
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	604 w.join()
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	605 print('end of main')
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	606 """
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	607 self.assertScriptHasOutput(script, "end of main\n")
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	608
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	609 @unittest.skipUnless(hasattr(os, 'fork'), "needs os.fork()")
71114 ff36b8cadfd6 Issue #11870: Skip test_threading.test_2_join_in_forked_process() on platforms Victor Stinner <victor.stinner@haypocalc.com> parents: 70447 diff changeset	610 @unittest.skipIf(sys.platform in platforms_to_skip, "due to known OS bug")
67341 3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	611 def test_5_clear_waiter_locks_to_avoid_crash(self):
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	612 # Check that a spawned thread that forks doesn't segfault on certain
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	613 # platforms, namely OS X. This used to happen if there was a waiter
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	614 # lock in the thread's condition variable's waiters list. Even though
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	615 # we know the lock will be held across the fork, it is not safe to
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	616 # release locks held across forks on all platforms, so releasing the
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	617 # waiter lock caused a segfault on OS X. Furthermore, since locks on
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	618 # OS X are (as of this writing) implemented with a mutex + condition
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	619 # variable instead of a semaphore, while we know that the Python-level
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	620 # lock will be acquired, we can't know if the internal mutex will be
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	621 # acquired at the time of the fork.
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	622
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	623 script = """if True:
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	624 import os, time, threading
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	625
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	626 start_fork = False
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	627
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	628 def worker():
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	629 # Wait until the main thread has attempted to join this thread
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	630 # before continuing.
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	631 while not start_fork:
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	632 time.sleep(0.01)
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	633 childpid = os.fork()
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	634 if childpid != 0:
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	635 # Parent process just waits for child.
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	636 (cpid, rc) = os.waitpid(childpid, 0)
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	637 assert cpid == childpid
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	638 assert rc == 0
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	639 print('end of worker thread')
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	640 else:
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	641 # Child process should just return.
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	642 pass
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	643
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	644 w = threading.Thread(target=worker)
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	645
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	646 # Stub out the private condition variable's _release_save method.
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	647 # This releases the condition's lock and flips the global that
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	648 # causes the worker to fork. At this point, the problematic waiter
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	649 # lock has been acquired once by the waiter and has been put onto
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	650 # the waiters list.
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	651 condition = w._block
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	652 orig_release_save = condition._release_save
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	653 def my_release_save():
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	654 global start_fork
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	655 orig_release_save()
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	656 # Waiter lock held here, condition lock released.
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	657 start_fork = True
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	658 condition._release_save = my_release_save
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	659
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	660 w.start()
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	661 w.join()
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	662 print('end of main thread')
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	663 """
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	664 output = "end of worker thread\nend of main thread\n"
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	665 self.assertScriptHasOutput(script, output)
3e1c3d2a8689 Merged revisions 87710 via svnmerge from Gregory P. Smith <greg@mad-scientist.com> parents: 66348 diff changeset	666
74729 c3649173d093 Issue #13817: After fork(), reinit the ad-hoc TLS implementation earlier to fix Charles-François Natali <neologix@free.fr> parents: 71114 diff changeset	667 @unittest.skipUnless(hasattr(os, 'fork'), "needs os.fork()")
74829 a47f97bd3c33 Skip test_threading.test_reinit_tls_after_fork() on platforms where fork() Charles-François Natali <neologix@free.fr> parents: 74729 diff changeset	668 @unittest.skipIf(sys.platform in platforms_to_skip, "due to known OS bug")
74729 c3649173d093 Issue #13817: After fork(), reinit the ad-hoc TLS implementation earlier to fix Charles-François Natali <neologix@free.fr> parents: 71114 diff changeset	669 def test_reinit_tls_after_fork(self):
c3649173d093 Issue #13817: After fork(), reinit the ad-hoc TLS implementation earlier to fix Charles-François Natali <neologix@free.fr> parents: 71114 diff changeset	670 # Issue #13817: fork() would deadlock in a multithreaded program with
c3649173d093 Issue #13817: After fork(), reinit the ad-hoc TLS implementation earlier to fix Charles-François Natali <neologix@free.fr> parents: 71114 diff changeset	671 # the ad-hoc TLS implementation.
c3649173d093 Issue #13817: After fork(), reinit the ad-hoc TLS implementation earlier to fix Charles-François Natali <neologix@free.fr> parents: 71114 diff changeset	672
c3649173d093 Issue #13817: After fork(), reinit the ad-hoc TLS implementation earlier to fix Charles-François Natali <neologix@free.fr> parents: 71114 diff changeset	673 def do_fork_and_wait():
c3649173d093 Issue #13817: After fork(), reinit the ad-hoc TLS implementation earlier to fix Charles-François Natali <neologix@free.fr> parents: 71114 diff changeset	674 # just fork a child process and wait it
c3649173d093 Issue #13817: After fork(), reinit the ad-hoc TLS implementation earlier to fix Charles-François Natali <neologix@free.fr> parents: 71114 diff changeset	675 pid = os.fork()
c3649173d093 Issue #13817: After fork(), reinit the ad-hoc TLS implementation earlier to fix Charles-François Natali <neologix@free.fr> parents: 71114 diff changeset	676 if pid > 0:
c3649173d093 Issue #13817: After fork(), reinit the ad-hoc TLS implementation earlier to fix Charles-François Natali <neologix@free.fr> parents: 71114 diff changeset	677 os.waitpid(pid, 0)
c3649173d093 Issue #13817: After fork(), reinit the ad-hoc TLS implementation earlier to fix Charles-François Natali <neologix@free.fr> parents: 71114 diff changeset	678 else:
c3649173d093 Issue #13817: After fork(), reinit the ad-hoc TLS implementation earlier to fix Charles-François Natali <neologix@free.fr> parents: 71114 diff changeset	679 os._exit(0)
c3649173d093 Issue #13817: After fork(), reinit the ad-hoc TLS implementation earlier to fix Charles-François Natali <neologix@free.fr> parents: 71114 diff changeset	680
c3649173d093 Issue #13817: After fork(), reinit the ad-hoc TLS implementation earlier to fix Charles-François Natali <neologix@free.fr> parents: 71114 diff changeset	681 # start a bunch of threads that will fork() child processes
c3649173d093 Issue #13817: After fork(), reinit the ad-hoc TLS implementation earlier to fix Charles-François Natali <neologix@free.fr> parents: 71114 diff changeset	682 threads = []
c3649173d093 Issue #13817: After fork(), reinit the ad-hoc TLS implementation earlier to fix Charles-François Natali <neologix@free.fr> parents: 71114 diff changeset	683 for i in range(16):
c3649173d093 Issue #13817: After fork(), reinit the ad-hoc TLS implementation earlier to fix Charles-François Natali <neologix@free.fr> parents: 71114 diff changeset	684 t = threading.Thread(target=do_fork_and_wait)
c3649173d093 Issue #13817: After fork(), reinit the ad-hoc TLS implementation earlier to fix Charles-François Natali <neologix@free.fr> parents: 71114 diff changeset	685 threads.append(t)
c3649173d093 Issue #13817: After fork(), reinit the ad-hoc TLS implementation earlier to fix Charles-François Natali <neologix@free.fr> parents: 71114 diff changeset	686 t.start()
c3649173d093 Issue #13817: After fork(), reinit the ad-hoc TLS implementation earlier to fix Charles-François Natali <neologix@free.fr> parents: 71114 diff changeset	687
c3649173d093 Issue #13817: After fork(), reinit the ad-hoc TLS implementation earlier to fix Charles-François Natali <neologix@free.fr> parents: 71114 diff changeset	688 for t in threads:
c3649173d093 Issue #13817: After fork(), reinit the ad-hoc TLS implementation earlier to fix Charles-François Natali <neologix@free.fr> parents: 71114 diff changeset	689 t.join()
c3649173d093 Issue #13817: After fork(), reinit the ad-hoc TLS implementation earlier to fix Charles-François Natali <neologix@free.fr> parents: 71114 diff changeset	690
47985 09beb96ba20d Apply patch for 874900: threading module can deadlock after fork Jesse Noller <jnoller@gmail.com> parents: 47598 diff changeset	691
56882 5f61bd3aa427 Suppress transient refleaks in test_threading. Antoine Pitrou <solipsis@pitrou.net> parents: 56686 diff changeset	692 class ThreadingExceptionTests(BaseTestCase):
41960 588686d4c2cf Patch #1731049: make threading.py use a proper "raise" when checking internal state, rather than assert statements (which get stripped out by -O). Collin Winter <collinw@gmail.com> parents: 39434 diff changeset	693 # A RuntimeError should be raised if Thread.start() is called
588686d4c2cf Patch #1731049: make threading.py use a proper "raise" when checking internal state, rather than assert statements (which get stripped out by -O). Collin Winter <collinw@gmail.com> parents: 39434 diff changeset	694 # multiple times.
588686d4c2cf Patch #1731049: make threading.py use a proper "raise" when checking internal state, rather than assert statements (which get stripped out by -O). Collin Winter <collinw@gmail.com> parents: 39434 diff changeset	695 def test_start_thread_again(self):
588686d4c2cf Patch #1731049: make threading.py use a proper "raise" when checking internal state, rather than assert statements (which get stripped out by -O). Collin Winter <collinw@gmail.com> parents: 39434 diff changeset	696 thread = threading.Thread()
588686d4c2cf Patch #1731049: make threading.py use a proper "raise" when checking internal state, rather than assert statements (which get stripped out by -O). Collin Winter <collinw@gmail.com> parents: 39434 diff changeset	697 thread.start()
588686d4c2cf Patch #1731049: make threading.py use a proper "raise" when checking internal state, rather than assert statements (which get stripped out by -O). Collin Winter <collinw@gmail.com> parents: 39434 diff changeset	698 self.assertRaises(RuntimeError, thread.start)
588686d4c2cf Patch #1731049: make threading.py use a proper "raise" when checking internal state, rather than assert statements (which get stripped out by -O). Collin Winter <collinw@gmail.com> parents: 39434 diff changeset	699
588686d4c2cf Patch #1731049: make threading.py use a proper "raise" when checking internal state, rather than assert statements (which get stripped out by -O). Collin Winter <collinw@gmail.com> parents: 39434 diff changeset	700 def test_joining_current_thread(self):
47598 c349f9fbc704 give the threading API PEP 8 names Benjamin Peterson <benjamin@python.org> parents: 47438 diff changeset	701 current_thread = threading.current_thread()
c349f9fbc704 give the threading API PEP 8 names Benjamin Peterson <benjamin@python.org> parents: 47438 diff changeset	702 self.assertRaises(RuntimeError, current_thread.join);
41960 588686d4c2cf Patch #1731049: make threading.py use a proper "raise" when checking internal state, rather than assert statements (which get stripped out by -O). Collin Winter <collinw@gmail.com> parents: 39434 diff changeset	703
588686d4c2cf Patch #1731049: make threading.py use a proper "raise" when checking internal state, rather than assert statements (which get stripped out by -O). Collin Winter <collinw@gmail.com> parents: 39434 diff changeset	704 def test_joining_inactive_thread(self):
588686d4c2cf Patch #1731049: make threading.py use a proper "raise" when checking internal state, rather than assert statements (which get stripped out by -O). Collin Winter <collinw@gmail.com> parents: 39434 diff changeset	705 thread = threading.Thread()
588686d4c2cf Patch #1731049: make threading.py use a proper "raise" when checking internal state, rather than assert statements (which get stripped out by -O). Collin Winter <collinw@gmail.com> parents: 39434 diff changeset	706 self.assertRaises(RuntimeError, thread.join)
588686d4c2cf Patch #1731049: make threading.py use a proper "raise" when checking internal state, rather than assert statements (which get stripped out by -O). Collin Winter <collinw@gmail.com> parents: 39434 diff changeset	707
588686d4c2cf Patch #1731049: make threading.py use a proper "raise" when checking internal state, rather than assert statements (which get stripped out by -O). Collin Winter <collinw@gmail.com> parents: 39434 diff changeset	708 def test_daemonize_active_thread(self):
588686d4c2cf Patch #1731049: make threading.py use a proper "raise" when checking internal state, rather than assert statements (which get stripped out by -O). Collin Winter <collinw@gmail.com> parents: 39434 diff changeset	709 thread = threading.Thread()
588686d4c2cf Patch #1731049: make threading.py use a proper "raise" when checking internal state, rather than assert statements (which get stripped out by -O). Collin Winter <collinw@gmail.com> parents: 39434 diff changeset	710 thread.start()
48288 e143a8f35241 backport threading property changes Benjamin Peterson <benjamin@python.org> parents: 48287 diff changeset	711 self.assertRaises(RuntimeError, setattr, thread, "daemon", True)
41960 588686d4c2cf Patch #1731049: make threading.py use a proper "raise" when checking internal state, rather than assert statements (which get stripped out by -O). Collin Winter <collinw@gmail.com> parents: 39434 diff changeset	712
588686d4c2cf Patch #1731049: make threading.py use a proper "raise" when checking internal state, rather than assert statements (which get stripped out by -O). Collin Winter <collinw@gmail.com> parents: 39434 diff changeset	713
57159 0eb56a8e6262 Merged revisions 76137,76172 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 57140 diff changeset	714 class LockTests(lock_tests.LockTests):
0eb56a8e6262 Merged revisions 76137,76172 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 57140 diff changeset	715 locktype = staticmethod(threading.Lock)
0eb56a8e6262 Merged revisions 76137,76172 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 57140 diff changeset	716
0eb56a8e6262 Merged revisions 76137,76172 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 57140 diff changeset	717 class RLockTests(lock_tests.RLockTests):
0eb56a8e6262 Merged revisions 76137,76172 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 57140 diff changeset	718 locktype = staticmethod(threading.RLock)
0eb56a8e6262 Merged revisions 76137,76172 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 57140 diff changeset	719
0eb56a8e6262 Merged revisions 76137,76172 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 57140 diff changeset	720 class EventTests(lock_tests.EventTests):
0eb56a8e6262 Merged revisions 76137,76172 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 57140 diff changeset	721 eventtype = staticmethod(threading.Event)
0eb56a8e6262 Merged revisions 76137,76172 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 57140 diff changeset	722
0eb56a8e6262 Merged revisions 76137,76172 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 57140 diff changeset	723 class ConditionAsRLockTests(lock_tests.RLockTests):
0eb56a8e6262 Merged revisions 76137,76172 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 57140 diff changeset	724 # An Condition uses an RLock by default and exports its API.
0eb56a8e6262 Merged revisions 76137,76172 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 57140 diff changeset	725 locktype = staticmethod(threading.Condition)
0eb56a8e6262 Merged revisions 76137,76172 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 57140 diff changeset	726
0eb56a8e6262 Merged revisions 76137,76172 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 57140 diff changeset	727 class ConditionTests(lock_tests.ConditionTests):
0eb56a8e6262 Merged revisions 76137,76172 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 57140 diff changeset	728 condtype = staticmethod(threading.Condition)
0eb56a8e6262 Merged revisions 76137,76172 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 57140 diff changeset	729
0eb56a8e6262 Merged revisions 76137,76172 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 57140 diff changeset	730 class SemaphoreTests(lock_tests.SemaphoreTests):
0eb56a8e6262 Merged revisions 76137,76172 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 57140 diff changeset	731 semtype = staticmethod(threading.Semaphore)
0eb56a8e6262 Merged revisions 76137,76172 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 57140 diff changeset	732
0eb56a8e6262 Merged revisions 76137,76172 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 57140 diff changeset	733 class BoundedSemaphoreTests(lock_tests.BoundedSemaphoreTests):
0eb56a8e6262 Merged revisions 76137,76172 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 57140 diff changeset	734 semtype = staticmethod(threading.BoundedSemaphore)
0eb56a8e6262 Merged revisions 76137,76172 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 57140 diff changeset	735
70447 ecf0ef85c72a Issue #9670: Increase the default stack size for secondary threads on Ned Deily <nad@acm.org> parents: 69228 diff changeset	736 @unittest.skipUnless(sys.platform == 'darwin', 'test macosx problem')
ecf0ef85c72a Issue #9670: Increase the default stack size for secondary threads on Ned Deily <nad@acm.org> parents: 69228 diff changeset	737 def test_recursion_limit(self):
ecf0ef85c72a Issue #9670: Increase the default stack size for secondary threads on Ned Deily <nad@acm.org> parents: 69228 diff changeset	738 # Issue 9670
ecf0ef85c72a Issue #9670: Increase the default stack size for secondary threads on Ned Deily <nad@acm.org> parents: 69228 diff changeset	739 # test that excessive recursion within a non-main thread causes
ecf0ef85c72a Issue #9670: Increase the default stack size for secondary threads on Ned Deily <nad@acm.org> parents: 69228 diff changeset	740 # an exception rather than crashing the interpreter on platforms
ecf0ef85c72a Issue #9670: Increase the default stack size for secondary threads on Ned Deily <nad@acm.org> parents: 69228 diff changeset	741 # like Mac OS X or FreeBSD which have small default stack sizes
ecf0ef85c72a Issue #9670: Increase the default stack size for secondary threads on Ned Deily <nad@acm.org> parents: 69228 diff changeset	742 # for threads
ecf0ef85c72a Issue #9670: Increase the default stack size for secondary threads on Ned Deily <nad@acm.org> parents: 69228 diff changeset	743 script = """if True:
ecf0ef85c72a Issue #9670: Increase the default stack size for secondary threads on Ned Deily <nad@acm.org> parents: 69228 diff changeset	744 import threading
ecf0ef85c72a Issue #9670: Increase the default stack size for secondary threads on Ned Deily <nad@acm.org> parents: 69228 diff changeset	745
ecf0ef85c72a Issue #9670: Increase the default stack size for secondary threads on Ned Deily <nad@acm.org> parents: 69228 diff changeset	746 def recurse():
ecf0ef85c72a Issue #9670: Increase the default stack size for secondary threads on Ned Deily <nad@acm.org> parents: 69228 diff changeset	747 return recurse()
ecf0ef85c72a Issue #9670: Increase the default stack size for secondary threads on Ned Deily <nad@acm.org> parents: 69228 diff changeset	748
ecf0ef85c72a Issue #9670: Increase the default stack size for secondary threads on Ned Deily <nad@acm.org> parents: 69228 diff changeset	749 def outer():
ecf0ef85c72a Issue #9670: Increase the default stack size for secondary threads on Ned Deily <nad@acm.org> parents: 69228 diff changeset	750 try:
ecf0ef85c72a Issue #9670: Increase the default stack size for secondary threads on Ned Deily <nad@acm.org> parents: 69228 diff changeset	751 recurse()
ecf0ef85c72a Issue #9670: Increase the default stack size for secondary threads on Ned Deily <nad@acm.org> parents: 69228 diff changeset	752 except RuntimeError:
ecf0ef85c72a Issue #9670: Increase the default stack size for secondary threads on Ned Deily <nad@acm.org> parents: 69228 diff changeset	753 pass
ecf0ef85c72a Issue #9670: Increase the default stack size for secondary threads on Ned Deily <nad@acm.org> parents: 69228 diff changeset	754
ecf0ef85c72a Issue #9670: Increase the default stack size for secondary threads on Ned Deily <nad@acm.org> parents: 69228 diff changeset	755 w = threading.Thread(target=outer)
ecf0ef85c72a Issue #9670: Increase the default stack size for secondary threads on Ned Deily <nad@acm.org> parents: 69228 diff changeset	756 w.start()
ecf0ef85c72a Issue #9670: Increase the default stack size for secondary threads on Ned Deily <nad@acm.org> parents: 69228 diff changeset	757 w.join()
ecf0ef85c72a Issue #9670: Increase the default stack size for secondary threads on Ned Deily <nad@acm.org> parents: 69228 diff changeset	758 print('end of main thread')
ecf0ef85c72a Issue #9670: Increase the default stack size for secondary threads on Ned Deily <nad@acm.org> parents: 69228 diff changeset	759 """
ecf0ef85c72a Issue #9670: Increase the default stack size for secondary threads on Ned Deily <nad@acm.org> parents: 69228 diff changeset	760 expected_output = "end of main thread\n"
ecf0ef85c72a Issue #9670: Increase the default stack size for secondary threads on Ned Deily <nad@acm.org> parents: 69228 diff changeset	761 p = subprocess.Popen([sys.executable, "-c", script],
ecf0ef85c72a Issue #9670: Increase the default stack size for secondary threads on Ned Deily <nad@acm.org> parents: 69228 diff changeset	762 stdout=subprocess.PIPE)
ecf0ef85c72a Issue #9670: Increase the default stack size for secondary threads on Ned Deily <nad@acm.org> parents: 69228 diff changeset	763 stdout, stderr = p.communicate()
ecf0ef85c72a Issue #9670: Increase the default stack size for secondary threads on Ned Deily <nad@acm.org> parents: 69228 diff changeset	764 data = stdout.decode().replace('\r', '')
ecf0ef85c72a Issue #9670: Increase the default stack size for secondary threads on Ned Deily <nad@acm.org> parents: 69228 diff changeset	765 self.assertEqual(p.returncode, 0, "Unexpected error")
ecf0ef85c72a Issue #9670: Increase the default stack size for secondary threads on Ned Deily <nad@acm.org> parents: 69228 diff changeset	766 self.assertEqual(data, expected_output)
57159 0eb56a8e6262 Merged revisions 76137,76172 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 57140 diff changeset	767
34362 cc27b249755d Converted to a unittest. Added checks that the bounded semaphore actually Tim Peters <tim.peters@gmail.com> parents: 24364 diff changeset	768 def test_main():
57159 0eb56a8e6262 Merged revisions 76137,76172 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 57140 diff changeset	769 test.test_support.run_unittest(LockTests, RLockTests, EventTests,
0eb56a8e6262 Merged revisions 76137,76172 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 57140 diff changeset	770 ConditionAsRLockTests, ConditionTests,
0eb56a8e6262 Merged revisions 76137,76172 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 57140 diff changeset	771 SemaphoreTests, BoundedSemaphoreTests,
0eb56a8e6262 Merged revisions 76137,76172 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 57140 diff changeset	772 ThreadTests,
47985 09beb96ba20d Apply patch for 874900: threading module can deadlock after fork Jesse Noller <jnoller@gmail.com> parents: 47598 diff changeset	773 ThreadJoinOnShutdown,
09beb96ba20d Apply patch for 874900: threading module can deadlock after fork Jesse Noller <jnoller@gmail.com> parents: 47598 diff changeset	774 ThreadingExceptionTests,
09beb96ba20d Apply patch for 874900: threading module can deadlock after fork Jesse Noller <jnoller@gmail.com> parents: 47598 diff changeset	775 )
34362 cc27b249755d Converted to a unittest. Added checks that the bounded semaphore actually Tim Peters <tim.peters@gmail.com> parents: 24364 diff changeset	776
cc27b249755d Converted to a unittest. Added checks that the bounded semaphore actually Tim Peters <tim.peters@gmail.com> parents: 24364 diff changeset	777 if __name__ == "__main__":
cc27b249755d Converted to a unittest. Added checks that the bounded semaphore actually Tim Peters <tim.peters@gmail.com> parents: 24364 diff changeset	778 test_main()

Mercurial > cpython-lru_cache-weakref

annotate Lib/test/test_threading.py @ 80661:94de77bd0b4b 2.7