Mercurial > cpython-lru_cache-weakref

annotate Lib/test/test_importlib/source/util.py @ 87690:66c1c9f32567 default tip

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Add use_weakref option to functools.lru_cache The use_weakref option makes lru_cache create weak references to the function arguments when creating the cache key, which prevents them from staying in the memory forever even after all other references to them are gone. This is a proof of concept for http://bugs.python.org/issue19859
author Radomir Dopieralski <devel@sheep.art.pl>
date Tue, 03 Dec 2013 16:02:48 +0100
parents 5e8b377942f7
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
51236
d85ef26efd10 Rename importlib.test.support to importlib.test.util.
Brett Cannon <bcannon@gmail.com>
parents: 51234
diff changeset
1 from .. import util
51234
4b3246c83e3f Split out support code that is specific to source tests out of
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
2 import contextlib
60648
7b69e630d237 PEP 3147
Barry Warsaw <barry@python.org>
parents: 57115
diff changeset
3 import errno
52083
c06f02a5e962 Introduce importlib.abc. The module contains various ABCs related to imports
Brett Cannon <bcannon@gmail.com>
parents: 52077
diff changeset
4 import functools
51234
4b3246c83e3f Split out support code that is specific to source tests out of
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
5 import os
4b3246c83e3f Split out support code that is specific to source tests out of
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
6 import os.path
4b3246c83e3f Split out support code that is specific to source tests out of
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
7 import sys
4b3246c83e3f Split out support code that is specific to source tests out of
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
8 import tempfile
51236
d85ef26efd10 Rename importlib.test.support to importlib.test.util.
Brett Cannon <bcannon@gmail.com>
parents: 51234
diff changeset
9 from test import support
51234
4b3246c83e3f Split out support code that is specific to source tests out of
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
10
4b3246c83e3f Split out support code that is specific to source tests out of
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
11
53996
0495bc47cd08 Tests for case-senstivity were not being skipped for darwin when installed on a
Brett Cannon <bcannon@gmail.com>
parents: 52083
diff changeset
12 def writes_bytecode_files(fxn):
0495bc47cd08 Tests for case-senstivity were not being skipped for darwin when installed on a
Brett Cannon <bcannon@gmail.com>
parents: 52083
diff changeset
13 """Decorator to protect sys.dont_write_bytecode from mutation and to skip
0495bc47cd08 Tests for case-senstivity were not being skipped for darwin when installed on a
Brett Cannon <bcannon@gmail.com>
parents: 52083
diff changeset
14 tests that require it to be set to False."""
0495bc47cd08 Tests for case-senstivity were not being skipped for darwin when installed on a
Brett Cannon <bcannon@gmail.com>
parents: 52083
diff changeset
15 if sys.dont_write_bytecode:
0495bc47cd08 Tests for case-senstivity were not being skipped for darwin when installed on a
Brett Cannon <bcannon@gmail.com>
parents: 52083
diff changeset
16 return lambda *args, **kwargs: None
52083
c06f02a5e962 Introduce importlib.abc. The module contains various ABCs related to imports
Brett Cannon <bcannon@gmail.com>
parents: 52077
diff changeset
17 @functools.wraps(fxn)
c06f02a5e962 Introduce importlib.abc. The module contains various ABCs related to imports
Brett Cannon <bcannon@gmail.com>
parents: 52077
diff changeset
18 def wrapper(*args, **kwargs):
c06f02a5e962 Introduce importlib.abc. The module contains various ABCs related to imports
Brett Cannon <bcannon@gmail.com>
parents: 52077
diff changeset
19 original = sys.dont_write_bytecode
c06f02a5e962 Introduce importlib.abc. The module contains various ABCs related to imports
Brett Cannon <bcannon@gmail.com>
parents: 52077
diff changeset
20 sys.dont_write_bytecode = False
53996
0495bc47cd08 Tests for case-senstivity were not being skipped for darwin when installed on a
Brett Cannon <bcannon@gmail.com>
parents: 52083
diff changeset
21 try:
0495bc47cd08 Tests for case-senstivity were not being skipped for darwin when installed on a
Brett Cannon <bcannon@gmail.com>
parents: 52083
diff changeset
22 to_return = fxn(*args, **kwargs)
0495bc47cd08 Tests for case-senstivity were not being skipped for darwin when installed on a
Brett Cannon <bcannon@gmail.com>
parents: 52083
diff changeset
23 finally:
0495bc47cd08 Tests for case-senstivity were not being skipped for darwin when installed on a
Brett Cannon <bcannon@gmail.com>
parents: 52083
diff changeset
24 sys.dont_write_bytecode = original
52083
c06f02a5e962 Introduce importlib.abc. The module contains various ABCs related to imports
Brett Cannon <bcannon@gmail.com>
parents: 52077
diff changeset
25 return to_return
c06f02a5e962 Introduce importlib.abc. The module contains various ABCs related to imports
Brett Cannon <bcannon@gmail.com>
parents: 52077
diff changeset
26 return wrapper
c06f02a5e962 Introduce importlib.abc. The module contains various ABCs related to imports
Brett Cannon <bcannon@gmail.com>
parents: 52077
diff changeset
27
c06f02a5e962 Introduce importlib.abc. The module contains various ABCs related to imports
Brett Cannon <bcannon@gmail.com>
parents: 52077
diff changeset
28
60648
7b69e630d237 PEP 3147
Barry Warsaw <barry@python.org>
parents: 57115
diff changeset
29 def ensure_bytecode_path(bytecode_path):
7b69e630d237 PEP 3147
Barry Warsaw <barry@python.org>
parents: 57115
diff changeset
30 """Ensure that the __pycache__ directory for PEP 3147 pyc file exists.
7b69e630d237 PEP 3147
Barry Warsaw <barry@python.org>
parents: 57115
diff changeset
31
7b69e630d237 PEP 3147
Barry Warsaw <barry@python.org>
parents: 57115
diff changeset
32 :param bytecode_path: File system path to PEP 3147 pyc file.
7b69e630d237 PEP 3147
Barry Warsaw <barry@python.org>
parents: 57115
diff changeset
33 """
7b69e630d237 PEP 3147
Barry Warsaw <barry@python.org>
parents: 57115
diff changeset
34 try:
7b69e630d237 PEP 3147
Barry Warsaw <barry@python.org>
parents: 57115
diff changeset
35 os.mkdir(os.path.dirname(bytecode_path))
7b69e630d237 PEP 3147
Barry Warsaw <barry@python.org>
parents: 57115
diff changeset
36 except OSError as error:
7b69e630d237 PEP 3147
Barry Warsaw <barry@python.org>
parents: 57115
diff changeset
37 if error.errno != errno.EEXIST:
7b69e630d237 PEP 3147
Barry Warsaw <barry@python.org>
parents: 57115
diff changeset
38 raise
51234
4b3246c83e3f Split out support code that is specific to source tests out of
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
39
4b3246c83e3f Split out support code that is specific to source tests out of
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
40
4b3246c83e3f Split out support code that is specific to source tests out of
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
41 @contextlib.contextmanager
4b3246c83e3f Split out support code that is specific to source tests out of
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
42 def create_modules(*names):
4b3246c83e3f Split out support code that is specific to source tests out of
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
43 """Temporarily create each named module with an attribute (named 'attr')
4b3246c83e3f Split out support code that is specific to source tests out of
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
44 that contains the name passed into the context manager that caused the
4b3246c83e3f Split out support code that is specific to source tests out of
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
45 creation of the module.
4b3246c83e3f Split out support code that is specific to source tests out of
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
46
57116
93c752875033 Merged revisions 76113-76114 via svnmerge from
Brett Cannon <bcannon@gmail.com>
parents: 53996
diff changeset
47 All files are created in a temporary directory returned by
93c752875033 Merged revisions 76113-76114 via svnmerge from
Brett Cannon <bcannon@gmail.com>
parents: 53996
diff changeset
48 tempfile.mkdtemp(). This directory is inserted at the beginning of
51234
4b3246c83e3f Split out support code that is specific to source tests out of
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
49 sys.path. When the context manager exits all created files (source and
4b3246c83e3f Split out support code that is specific to source tests out of
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
50 bytecode) are explicitly deleted.
4b3246c83e3f Split out support code that is specific to source tests out of
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
51
4b3246c83e3f Split out support code that is specific to source tests out of
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
52 No magic is performed when creating packages! This means that if you create
4b3246c83e3f Split out support code that is specific to source tests out of
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
53 a module within a package you must also create the package's __init__ as
4b3246c83e3f Split out support code that is specific to source tests out of
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
54 well.
4b3246c83e3f Split out support code that is specific to source tests out of
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
55
4b3246c83e3f Split out support code that is specific to source tests out of
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
56 """
4b3246c83e3f Split out support code that is specific to source tests out of
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
57 source = 'attr = {0!r}'
4b3246c83e3f Split out support code that is specific to source tests out of
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
58 created_paths = []
4b3246c83e3f Split out support code that is specific to source tests out of
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
59 mapping = {}
57116
93c752875033 Merged revisions 76113-76114 via svnmerge from
Brett Cannon <bcannon@gmail.com>
parents: 53996
diff changeset
60 state_manager = None
93c752875033 Merged revisions 76113-76114 via svnmerge from
Brett Cannon <bcannon@gmail.com>
parents: 53996
diff changeset
61 uncache_manager = None
51234
4b3246c83e3f Split out support code that is specific to source tests out of
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
62 try:
57116
93c752875033 Merged revisions 76113-76114 via svnmerge from
Brett Cannon <bcannon@gmail.com>
parents: 53996
diff changeset
63 temp_dir = tempfile.mkdtemp()
51234
4b3246c83e3f Split out support code that is specific to source tests out of
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
64 mapping['.root'] = temp_dir
4b3246c83e3f Split out support code that is specific to source tests out of
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
65 import_names = set()
4b3246c83e3f Split out support code that is specific to source tests out of
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
66 for name in names:
4b3246c83e3f Split out support code that is specific to source tests out of
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
67 if not name.endswith('__init__'):
4b3246c83e3f Split out support code that is specific to source tests out of
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
68 import_name = name
4b3246c83e3f Split out support code that is specific to source tests out of
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
69 else:
4b3246c83e3f Split out support code that is specific to source tests out of
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
70 import_name = name[:-len('.__init__')]
4b3246c83e3f Split out support code that is specific to source tests out of
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
71 import_names.add(import_name)
4b3246c83e3f Split out support code that is specific to source tests out of
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
72 if import_name in sys.modules:
4b3246c83e3f Split out support code that is specific to source tests out of
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
73 del sys.modules[import_name]
4b3246c83e3f Split out support code that is specific to source tests out of
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
74 name_parts = name.split('.')
4b3246c83e3f Split out support code that is specific to source tests out of
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
75 file_path = temp_dir
4b3246c83e3f Split out support code that is specific to source tests out of
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
76 for directory in name_parts[:-1]:
4b3246c83e3f Split out support code that is specific to source tests out of
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
77 file_path = os.path.join(file_path, directory)
4b3246c83e3f Split out support code that is specific to source tests out of
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
78 if not os.path.exists(file_path):
4b3246c83e3f Split out support code that is specific to source tests out of
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
79 os.mkdir(file_path)
4b3246c83e3f Split out support code that is specific to source tests out of
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
80 created_paths.append(file_path)
4b3246c83e3f Split out support code that is specific to source tests out of
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
81 file_path = os.path.join(file_path, name_parts[-1] + '.py')
4b3246c83e3f Split out support code that is specific to source tests out of
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
82 with open(file_path, 'w') as file:
4b3246c83e3f Split out support code that is specific to source tests out of
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
83 file.write(source.format(name))
4b3246c83e3f Split out support code that is specific to source tests out of
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
84 created_paths.append(file_path)
4b3246c83e3f Split out support code that is specific to source tests out of
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
85 mapping[name] = file_path
4b3246c83e3f Split out support code that is specific to source tests out of
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
86 uncache_manager = util.uncache(*import_names)
4b3246c83e3f Split out support code that is specific to source tests out of
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
87 uncache_manager.__enter__()
4b3246c83e3f Split out support code that is specific to source tests out of
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
88 state_manager = util.import_state(path=[temp_dir])
4b3246c83e3f Split out support code that is specific to source tests out of
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
89 state_manager.__enter__()
4b3246c83e3f Split out support code that is specific to source tests out of
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
90 yield mapping
4b3246c83e3f Split out support code that is specific to source tests out of
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
91 finally:
57116
93c752875033 Merged revisions 76113-76114 via svnmerge from
Brett Cannon <bcannon@gmail.com>
parents: 53996
diff changeset
92 if state_manager is not None:
93c752875033 Merged revisions 76113-76114 via svnmerge from
Brett Cannon <bcannon@gmail.com>
parents: 53996
diff changeset
93 state_manager.__exit__(None, None, None)
93c752875033 Merged revisions 76113-76114 via svnmerge from
Brett Cannon <bcannon@gmail.com>
parents: 53996
diff changeset
94 if uncache_manager is not None:
93c752875033 Merged revisions 76113-76114 via svnmerge from
Brett Cannon <bcannon@gmail.com>
parents: 53996
diff changeset
95 uncache_manager.__exit__(None, None, None)
93c752875033 Merged revisions 76113-76114 via svnmerge from
Brett Cannon <bcannon@gmail.com>
parents: 53996
diff changeset
96 support.rmtree(temp_dir)