Mercurial > cpython-lru_cache-weakref

annotate Lib/test/fork_wait.py @ 87690:66c1c9f32567 default tip

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Add use_weakref option to functools.lru_cache The use_weakref option makes lru_cache create weak references to the function arguments when creating the cache key, which prevents them from staying in the memory forever even after all other references to them are gone. This is a proof of concept for http://bugs.python.org/issue19859
author Radomir Dopieralski <devel@sheep.art.pl>
date Tue, 03 Dec 2013 16:02:48 +0100
parents 7d69d04522e3
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
diff changeset
1 """This test case provides support for checking forking and wait behavior.
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
diff changeset
2
61073
493bb4585e38 Merged revisions 80552-80556,80564-80566,80568-80571 via svnmerge from
Victor Stinner <victor.stinner@haypocalc.com>
parents: 47266
diff changeset
3 To test different wait behavior, override the wait_impl method.
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
diff changeset
4
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
diff changeset
5 We want fork1() semantics -- only the forking thread survives in the
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
diff changeset
6 child after a fork().
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
diff changeset
7
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
diff changeset
8 On some systems (e.g. Solaris without posix threads) we find that all
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
diff changeset
9 active threads survive in the child after a fork(); this is an error.
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
diff changeset
10 """
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
diff changeset
11
61073
493bb4585e38 Merged revisions 80552-80556,80564-80566,80568-80571 via svnmerge from
Victor Stinner <victor.stinner@haypocalc.com>
parents: 47266
diff changeset
12 import os, sys, time, unittest
493bb4585e38 Merged revisions 80552-80556,80564-80566,80568-80571 via svnmerge from
Victor Stinner <victor.stinner@haypocalc.com>
parents: 47266
diff changeset
13 import test.support as support
493bb4585e38 Merged revisions 80552-80556,80564-80566,80568-80571 via svnmerge from
Victor Stinner <victor.stinner@haypocalc.com>
parents: 47266
diff changeset
14 _thread = support.import_module('_thread')
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
diff changeset
15
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
diff changeset
16 LONGSLEEP = 2
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
diff changeset
17 SHORTSLEEP = 0.5
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
diff changeset
18 NUM_THREADS = 4
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
diff changeset
19
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
diff changeset
20 class ForkWait(unittest.TestCase):
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
diff changeset
21
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
diff changeset
22 def setUp(self):
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
diff changeset
23 self.alive = {}
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
diff changeset
24 self.stop = 0
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
diff changeset
25
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
diff changeset
26 def f(self, id):
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
diff changeset
27 while not self.stop:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
diff changeset
28 self.alive[id] = os.getpid()
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
diff changeset
29 try:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
diff changeset
30 time.sleep(SHORTSLEEP)
81039
7d69d04522e3 Replace IOError with OSError (#16715)
Andrew Svetlov <andrew.svetlov@gmail.com>
parents: 69991
diff changeset
31 except OSError:
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
diff changeset
32 pass
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
diff changeset
33
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
diff changeset
34 def wait_impl(self, cpid):
39442
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
35 for i in range(10):
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
36 # waitpid() shouldn't hang, but some of the buildbots seem to hang
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
37 # in the forking tests. This is an attempt to fix the problem.
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
38 spid, status = os.waitpid(cpid, os.WNOHANG)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
39 if spid == cpid:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
40 break
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
41 time.sleep(2 * SHORTSLEEP)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
42
66340
2beb25369289 Merged revisions 86596 via svnmerge from
Ezio Melotti <ezio.melotti@gmail.com>
parents: 47266
diff changeset
43 self.assertEqual(spid, cpid)
2beb25369289 Merged revisions 86596 via svnmerge from
Ezio Melotti <ezio.melotti@gmail.com>
parents: 47266
diff changeset
44 self.assertEqual(status, 0, "cause = %d, exit = %d" % (status&0xff, status>>8))
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
diff changeset
45
69991
9e473917cbfb Issue #12041: Make test_wait3 more robust.
Antoine Pitrou <solipsis@pitrou.net>
parents: 68257
diff changeset
46 @support.reap_threads
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
diff changeset
47 def test_wait(self):
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
diff changeset
48 for i in range(NUM_THREADS):
47266
7fe3a8648ce2 Rename thread to _thread and dummy_thread to _dummy_thread. Issue #2875.
Georg Brandl <georg@python.org>
parents: 45471
diff changeset
49 _thread.start_new(self.f, (i,))
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
diff changeset
50
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
diff changeset
51 time.sleep(LONGSLEEP)
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
diff changeset
52
40809
f340cb045bf9 - PEP 3106: dict.iterkeys(), .iteritems(), .itervalues() are now gone;
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
53 a = sorted(self.alive.keys())
66340
2beb25369289 Merged revisions 86596 via svnmerge from
Ezio Melotti <ezio.melotti@gmail.com>
parents: 47266
diff changeset
54 self.assertEqual(a, list(range(NUM_THREADS)))
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
diff changeset
55
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
diff changeset
56 prefork_lives = self.alive.copy()
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
diff changeset
57
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
diff changeset
58 if sys.platform in ['unixware7']:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
diff changeset
59 cpid = os.fork1()
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
diff changeset
60 else:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
diff changeset
61 cpid = os.fork()
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
diff changeset
62
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
diff changeset
63 if cpid == 0:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
diff changeset
64 # Child
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
diff changeset
65 time.sleep(LONGSLEEP)
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
diff changeset
66 n = 0
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
diff changeset
67 for key in self.alive:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
diff changeset
68 if self.alive[key] != prefork_lives[key]:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
diff changeset
69 n += 1
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
diff changeset
70 os._exit(n)
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
diff changeset
71 else:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
diff changeset
72 # Parent
69991
9e473917cbfb Issue #12041: Make test_wait3 more robust.
Antoine Pitrou <solipsis@pitrou.net>
parents: 68257
diff changeset
73 try:
9e473917cbfb Issue #12041: Make test_wait3 more robust.
Antoine Pitrou <solipsis@pitrou.net>
parents: 68257
diff changeset
74 self.wait_impl(cpid)
9e473917cbfb Issue #12041: Make test_wait3 more robust.
Antoine Pitrou <solipsis@pitrou.net>
parents: 68257
diff changeset
75 finally:
9e473917cbfb Issue #12041: Make test_wait3 more robust.
Antoine Pitrou <solipsis@pitrou.net>
parents: 68257
diff changeset
76 # Tell threads to die
9e473917cbfb Issue #12041: Make test_wait3 more robust.
Antoine Pitrou <solipsis@pitrou.net>
parents: 68257
diff changeset
77 self.stop = 1