Mercurial > cpython-lru_cache-weakref

annotate Lib/importlib/_bootstrap.py @ 86521:bb2affc1e317

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
cleanup the construction of __qualname__ (closes #19301 again)
author Benjamin Peterson <benjamin@python.org>
date Sun, 20 Oct 2013 17:50:28 -0400
parents 35b384ed594b
children 88c3a1a3c2ff
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
1 """Core implementation of import.
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
2
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
3 This module is NOT meant to be directly imported! It has been designed such
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
4 that it can be bootstrapped into Python as the implementation of import. As
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
5 such it requires the injection of specific modules and attributes in order to
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
6 work. One should use importlib as the public-facing version of this module.
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
7
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
8 """
76549
acfdf46b8de1 Issue #14605 and #14642:
Marc-Andre Lemburg <mal@egenix.com>
parents: 76548
diff changeset
9 #
acfdf46b8de1 Issue #14605 and #14642:
Marc-Andre Lemburg <mal@egenix.com>
parents: 76548
diff changeset
10 # IMPORTANT: Whenever making changes to this module, be sure to run
acfdf46b8de1 Issue #14605 and #14642:
Marc-Andre Lemburg <mal@egenix.com>
parents: 76548
diff changeset
11 # a top-level make in order to get the frozen version of the module
83960
3a1976680717 Introduce importlib.util.ModuleManager which is a context manager to
Brett Cannon <brett@python.org>
parents: 83791
diff changeset
12 # update. Not doing so will result in the Makefile to fail for
76549
acfdf46b8de1 Issue #14605 and #14642:
Marc-Andre Lemburg <mal@egenix.com>
parents: 76548
diff changeset
13 # all others who don't have a ./python around to freeze the module
acfdf46b8de1 Issue #14605 and #14642:
Marc-Andre Lemburg <mal@egenix.com>
parents: 76548
diff changeset
14 # in the early stages of compilation.
acfdf46b8de1 Issue #14605 and #14642:
Marc-Andre Lemburg <mal@egenix.com>
parents: 76548
diff changeset
15 #
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
16
76449
163360370ad2 Make path manipulation more robust for platforms with alternative path
Brett Cannon <brett@python.org>
parents: 76445
diff changeset
17 # See importlib._setup() for what is injected into the global namespace.
163360370ad2 Make path manipulation more robust for platforms with alternative path
Brett Cannon <brett@python.org>
parents: 76445
diff changeset
18
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
19 # When editing this code be aware that code executed at import time CANNOT
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
20 # reference any injected objects! This includes not only global code but also
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
21 # anything specified at the class level.
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
22
52140
1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap.
Brett Cannon <bcannon@gmail.com>
parents: 52139
diff changeset
23 # Bootstrap-related code ######################################################
1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap.
Brett Cannon <bcannon@gmail.com>
parents: 52139
diff changeset
24
76753
59b8a7fcb047 Properly mark names in importlib._bootstrap as private.
Brett Cannon <brett@python.org>
parents: 76586
diff changeset
25 _CASE_INSENSITIVE_PLATFORMS = 'win', 'cygwin', 'darwin'
74643
ecf4a7bb8807 Relocate importlib._case_ok to importlib._bootstrap.
Brett Cannon <brett@python.org>
parents: 74630
diff changeset
26
ecf4a7bb8807 Relocate importlib._case_ok to importlib._bootstrap.
Brett Cannon <brett@python.org>
parents: 74630
diff changeset
27
75055
b8576719fe71 _relax_case -> _make_relax_case
Antoine Pitrou <solipsis@pitrou.net>
parents: 75048
diff changeset
28 def _make_relax_case():
76753
59b8a7fcb047 Properly mark names in importlib._bootstrap as private.
Brett Cannon <brett@python.org>
parents: 76586
diff changeset
29 if sys.platform.startswith(_CASE_INSENSITIVE_PLATFORMS):
75048
bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function.
Antoine Pitrou <solipsis@pitrou.net>
parents: 75045
diff changeset
30 def _relax_case():
75070
defd944af91b put docstrings on functions
Benjamin Peterson <benjamin@python.org>
parents: 75055
diff changeset
31 """True if filenames must be checked case-insensitively."""
85518
7801ef4a4ce3 Issue #16826: Revert fix while Windows issues are being worked out.
Meador Inge <meadori@gmail.com>
parents: 85515
diff changeset
32 return b'PYTHONCASEOK' in _os.environ
74643
ecf4a7bb8807 Relocate importlib._case_ok to importlib._bootstrap.
Brett Cannon <brett@python.org>
parents: 74630
diff changeset
33 else:
75048
bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function.
Antoine Pitrou <solipsis@pitrou.net>
parents: 75045
diff changeset
34 def _relax_case():
75070
defd944af91b put docstrings on functions
Benjamin Peterson <benjamin@python.org>
parents: 75055
diff changeset
35 """True if filenames must be checked case-insensitively."""
75048
bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function.
Antoine Pitrou <solipsis@pitrou.net>
parents: 75045
diff changeset
36 return False
bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function.
Antoine Pitrou <solipsis@pitrou.net>
parents: 75045
diff changeset
37 return _relax_case
74643
ecf4a7bb8807 Relocate importlib._case_ok to importlib._bootstrap.
Brett Cannon <brett@python.org>
parents: 74630
diff changeset
38
ecf4a7bb8807 Relocate importlib._case_ok to importlib._bootstrap.
Brett Cannon <brett@python.org>
parents: 74630
diff changeset
39
74630
7d8f016784cd Move some code from importlib.__init__ to importlib._bootstrap that
Brett Cannon <brett@python.org>
parents: 74449
diff changeset
40 def _w_long(x):
83963
185a0e649fb8 Issue #18070: importlib.util.module_for_loader() now sets __loader__
Brett Cannon <brett@python.org>
parents: 83960
diff changeset
41 """Convert a 32-bit integer to little-endian."""
82385
d98a82f4c9bd Issue #17220: two fixes for changeset 2528e4aea338.
Brett Cannon <brett@python.org>
parents: 82382
diff changeset
42 return (int(x) & 0xFFFFFFFF).to_bytes(4, 'little')
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
43
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
44
74630
7d8f016784cd Move some code from importlib.__init__ to importlib._bootstrap that
Brett Cannon <brett@python.org>
parents: 74449
diff changeset
45 def _r_long(int_bytes):
83963
185a0e649fb8 Issue #18070: importlib.util.module_for_loader() now sets __loader__
Brett Cannon <brett@python.org>
parents: 83960
diff changeset
46 """Convert 4 bytes in little-endian to an integer."""
82382
2528e4aea338 Issue #17220: Little cleanup of _bootstrap.py.
Serhiy Storchaka <storchaka@gmail.com>
parents: 82237
diff changeset
47 return int.from_bytes(int_bytes, 'little')
74630
7d8f016784cd Move some code from importlib.__init__ to importlib._bootstrap that
Brett Cannon <brett@python.org>
parents: 74449
diff changeset
48
7d8f016784cd Move some code from importlib.__init__ to importlib._bootstrap that
Brett Cannon <brett@python.org>
parents: 74449
diff changeset
49
76472
4e853913054c Issue #13959: Continue to try to accomodate altsep in importlib by not
Brett Cannon <brett@python.org>
parents: 76466
diff changeset
50 def _path_join(*path_parts):
76449
163360370ad2 Make path manipulation more robust for platforms with alternative path
Brett Cannon <brett@python.org>
parents: 76445
diff changeset
51 """Replacement for os.path.join()."""
82382
2528e4aea338 Issue #17220: Little cleanup of _bootstrap.py.
Serhiy Storchaka <storchaka@gmail.com>
parents: 82237
diff changeset
52 return path_sep.join([part.rstrip(path_separators)
2528e4aea338 Issue #17220: Little cleanup of _bootstrap.py.
Serhiy Storchaka <storchaka@gmail.com>
parents: 82237
diff changeset
53 for part in path_parts if part])
74630
7d8f016784cd Move some code from importlib.__init__ to importlib._bootstrap that
Brett Cannon <brett@python.org>
parents: 74449
diff changeset
54
76449
163360370ad2 Make path manipulation more robust for platforms with alternative path
Brett Cannon <brett@python.org>
parents: 76445
diff changeset
55
163360370ad2 Make path manipulation more robust for platforms with alternative path
Brett Cannon <brett@python.org>
parents: 76445
diff changeset
56 def _path_split(path):
163360370ad2 Make path manipulation more robust for platforms with alternative path
Brett Cannon <brett@python.org>
parents: 76445
diff changeset
57 """Replacement for os.path.split()."""
82382
2528e4aea338 Issue #17220: Little cleanup of _bootstrap.py.
Serhiy Storchaka <storchaka@gmail.com>
parents: 82237
diff changeset
58 if len(path_separators) == 1:
2528e4aea338 Issue #17220: Little cleanup of _bootstrap.py.
Serhiy Storchaka <storchaka@gmail.com>
parents: 82237
diff changeset
59 front, _, tail = path.rpartition(path_sep)
2528e4aea338 Issue #17220: Little cleanup of _bootstrap.py.
Serhiy Storchaka <storchaka@gmail.com>
parents: 82237
diff changeset
60 return front, tail
76449
163360370ad2 Make path manipulation more robust for platforms with alternative path
Brett Cannon <brett@python.org>
parents: 76445
diff changeset
61 for x in reversed(path):
163360370ad2 Make path manipulation more robust for platforms with alternative path
Brett Cannon <brett@python.org>
parents: 76445
diff changeset
62 if x in path_separators:
82385
d98a82f4c9bd Issue #17220: two fixes for changeset 2528e4aea338.
Brett Cannon <brett@python.org>
parents: 82382
diff changeset
63 front, tail = path.rsplit(x, maxsplit=1)
82382
2528e4aea338 Issue #17220: Little cleanup of _bootstrap.py.
Serhiy Storchaka <storchaka@gmail.com>
parents: 82237
diff changeset
64 return front, tail
2528e4aea338 Issue #17220: Little cleanup of _bootstrap.py.
Serhiy Storchaka <storchaka@gmail.com>
parents: 82237
diff changeset
65 return '', path
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
66
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
67
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
68 def _path_is_mode_type(path, mode):
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
69 """Test whether the path is the specified mode type."""
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
70 try:
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
71 stat_info = _os.stat(path)
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
72 except OSError:
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
73 return False
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
74 return (stat_info.st_mode & 0o170000) == mode
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
75
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
76
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
77 # XXX Could also expose Modules/getpath.c:isfile()
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
78 def _path_isfile(path):
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
79 """Replacement for os.path.isfile."""
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
80 return _path_is_mode_type(path, 0o100000)
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
81
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
82
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
83 # XXX Could also expose Modules/getpath.c:isdir()
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
84 def _path_isdir(path):
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
85 """Replacement for os.path.isdir."""
62756
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib.
Brett Cannon <bcannon@gmail.com>
parents: 62587
diff changeset
86 if not path:
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib.
Brett Cannon <bcannon@gmail.com>
parents: 62587
diff changeset
87 path = _os.getcwd()
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
88 return _path_is_mode_type(path, 0o040000)
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
89
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
90
78722
3a831a0a29c4 Close #2501: Permission bits are once again correctly copied from the source file to the cached bytecode file. Test by Eric Snow.
Nick Coghlan <ncoghlan@gmail.com>
parents: 78663
diff changeset
91 def _write_atomic(path, data, mode=0o666):
75185
27d31f0c4ad5 Issue #14077: importlib: Fix regression introduced by de6703671386.
Charles-François Natali <neologix@free.fr>
parents: 75082
diff changeset
92 """Best-effort function to write data to a path atomically.
27d31f0c4ad5 Issue #14077: importlib: Fix regression introduced by de6703671386.
Charles-François Natali <neologix@free.fr>
parents: 75082
diff changeset
93 Be prepared to handle a FileExistsError if concurrent writing of the
27d31f0c4ad5 Issue #14077: importlib: Fix regression introduced by de6703671386.
Charles-François Natali <neologix@free.fr>
parents: 75082
diff changeset
94 temporary file is attempted."""
73573
b75b41237380 Issue #13392: Writing a pyc file should now be atomic under Windows as well.
Antoine Pitrou <solipsis@pitrou.net>
parents: 73477
diff changeset
95 # id() is used to generate a pseudo-random filename.
b75b41237380 Issue #13392: Writing a pyc file should now be atomic under Windows as well.
Antoine Pitrou <solipsis@pitrou.net>
parents: 73477
diff changeset
96 path_tmp = '{}.{}'.format(path, id(path))
78722
3a831a0a29c4 Close #2501: Permission bits are once again correctly copied from the source file to the cached bytecode file. Test by Eric Snow.
Nick Coghlan <ncoghlan@gmail.com>
parents: 78663
diff changeset
97 fd = _os.open(path_tmp,
3a831a0a29c4 Close #2501: Permission bits are once again correctly copied from the source file to the cached bytecode file. Test by Eric Snow.
Nick Coghlan <ncoghlan@gmail.com>
parents: 78663
diff changeset
98 _os.O_EXCL | _os.O_CREAT | _os.O_WRONLY, mode & 0o666)
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
99 try:
74998
de6703671386 Have importlib use os.replace() for atomic renaming.
Brett Cannon <brett@python.org>
parents: 74994
diff changeset
100 # We first write data to a temporary file, and then use os.replace() to
de6703671386 Have importlib use os.replace() for atomic renaming.
Brett Cannon <brett@python.org>
parents: 74994
diff changeset
101 # perform an atomic rename.
73573
b75b41237380 Issue #13392: Writing a pyc file should now be atomic under Windows as well.
Antoine Pitrou <solipsis@pitrou.net>
parents: 73477
diff changeset
102 with _io.FileIO(fd, 'wb') as file:
b75b41237380 Issue #13392: Writing a pyc file should now be atomic under Windows as well.
Antoine Pitrou <solipsis@pitrou.net>
parents: 73477
diff changeset
103 file.write(data)
74998
de6703671386 Have importlib use os.replace() for atomic renaming.
Brett Cannon <brett@python.org>
parents: 74994
diff changeset
104 _os.replace(path_tmp, path)
73573
b75b41237380 Issue #13392: Writing a pyc file should now be atomic under Windows as well.
Antoine Pitrou <solipsis@pitrou.net>
parents: 73477
diff changeset
105 except OSError:
b75b41237380 Issue #13392: Writing a pyc file should now be atomic under Windows as well.
Antoine Pitrou <solipsis@pitrou.net>
parents: 73477
diff changeset
106 try:
b75b41237380 Issue #13392: Writing a pyc file should now be atomic under Windows as well.
Antoine Pitrou <solipsis@pitrou.net>
parents: 73477
diff changeset
107 _os.unlink(path_tmp)
72960
c16063765d3a Issue #13146: Writing a pyc file is now atomic under POSIX.
Antoine Pitrou <solipsis@pitrou.net>
parents: 68887
diff changeset
108 except OSError:
73573
b75b41237380 Issue #13392: Writing a pyc file should now be atomic under Windows as well.
Antoine Pitrou <solipsis@pitrou.net>
parents: 73477
diff changeset
109 pass
b75b41237380 Issue #13392: Writing a pyc file should now be atomic under Windows as well.
Antoine Pitrou <solipsis@pitrou.net>
parents: 73477
diff changeset
110 raise
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
111
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
112
52139
2e8e897a1244 Make utility code in importlib._bootstrap private.
Brett Cannon <bcannon@gmail.com>
parents: 52115
diff changeset
113 def _wrap(new, old):
77964
37850f083af5 Minor refactoring in importlib._bootstrap, and fix the '_wrap' docstring.
Florent Xicluna <florent.xicluna@gmail.com>
parents: 77917
diff changeset
114 """Simple substitute for functools.update_wrapper."""
73977
54a77c556d9a Issue #13593: updating the importlib utility decorators for __qualname__.
Meador Inge <meadori@gmail.com>
parents: 73974
diff changeset
115 for replace in ['__module__', '__name__', '__qualname__', '__doc__']:
74838
7a6fd7cd16e9 Don't fail in the face of a lacking attribute when wrapping a
Brett Cannon <brett@python.org>
parents: 74643
diff changeset
116 if hasattr(old, replace):
7a6fd7cd16e9 Don't fail in the face of a lacking attribute when wrapping a
Brett Cannon <brett@python.org>
parents: 74643
diff changeset
117 setattr(new, replace, getattr(old, replace))
51433
43d3a95ed662 Create a simple substitute for functools.wraps to use in importlib._bootstrap.
Brett Cannon <bcannon@gmail.com>
parents: 51430
diff changeset
118 new.__dict__.update(old.__dict__)
43d3a95ed662 Create a simple substitute for functools.wraps to use in importlib._bootstrap.
Brett Cannon <bcannon@gmail.com>
parents: 51430
diff changeset
119
43d3a95ed662 Create a simple substitute for functools.wraps to use in importlib._bootstrap.
Brett Cannon <bcannon@gmail.com>
parents: 51430
diff changeset
120
76753
59b8a7fcb047 Properly mark names in importlib._bootstrap as private.
Brett Cannon <brett@python.org>
parents: 76586
diff changeset
121 _code_type = type(_wrap.__code__)
62756
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib.
Brett Cannon <bcannon@gmail.com>
parents: 62587
diff changeset
122
76331
d777f854a66e Issue #13959: Rename imp to _imp and add Lib/imp.py and begin
Brett Cannon <brett@python.org>
parents: 76330
diff changeset
123
77013
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
124 # Module-level locking ########################################################
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
125
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
126 # A dict mapping module names to weakrefs of _ModuleLock instances
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
127 _module_locks = {}
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
128 # A dict mapping thread ids to _ModuleLock instances
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
129 _blocking_on = {}
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
130
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
131
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
132 class _DeadlockError(RuntimeError):
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
133 pass
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
134
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
135
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
136 class _ModuleLock:
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
137 """A recursive lock implementation which is able to detect deadlocks
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
138 (e.g. thread 1 trying to take locks A then B, and thread 2 trying to
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
139 take locks B then A).
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
140 """
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
141
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
142 def __init__(self, name):
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
143 self.lock = _thread.allocate_lock()
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
144 self.wakeup = _thread.allocate_lock()
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
145 self.name = name
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
146 self.owner = None
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
147 self.count = 0
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
148 self.waiters = 0
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
149
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
150 def has_deadlock(self):
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
151 # Deadlock avoidance for concurrent circular imports.
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
152 me = _thread.get_ident()
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
153 tid = self.owner
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
154 while True:
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
155 lock = _blocking_on.get(tid)
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
156 if lock is None:
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
157 return False
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
158 tid = lock.owner
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
159 if tid == me:
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
160 return True
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
161
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
162 def acquire(self):
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
163 """
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
164 Acquire the module lock. If a potential deadlock is detected,
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
165 a _DeadlockError is raised.
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
166 Otherwise, the lock is always acquired and True is returned.
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
167 """
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
168 tid = _thread.get_ident()
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
169 _blocking_on[tid] = self
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
170 try:
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
171 while True:
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
172 with self.lock:
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
173 if self.count == 0 or self.owner == tid:
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
174 self.owner = tid
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
175 self.count += 1
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
176 return True
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
177 if self.has_deadlock():
84582
33d379c2bb79 Issue #18415: Normalize what type of quotes are used with string
Brett Cannon <brett@python.org>
parents: 84468
diff changeset
178 raise _DeadlockError('deadlock detected by %r' % self)
77013
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
179 if self.wakeup.acquire(False):
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
180 self.waiters += 1
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
181 # Wait for a release() call
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
182 self.wakeup.acquire()
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
183 self.wakeup.release()
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
184 finally:
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
185 del _blocking_on[tid]
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
186
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
187 def release(self):
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
188 tid = _thread.get_ident()
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
189 with self.lock:
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
190 if self.owner != tid:
84582
33d379c2bb79 Issue #18415: Normalize what type of quotes are used with string
Brett Cannon <brett@python.org>
parents: 84468
diff changeset
191 raise RuntimeError('cannot release un-acquired lock')
77013
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
192 assert self.count > 0
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
193 self.count -= 1
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
194 if self.count == 0:
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
195 self.owner = None
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
196 if self.waiters:
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
197 self.waiters -= 1
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
198 self.wakeup.release()
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
199
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
200 def __repr__(self):
84582
33d379c2bb79 Issue #18415: Normalize what type of quotes are used with string
Brett Cannon <brett@python.org>
parents: 84468
diff changeset
201 return '_ModuleLock({!r}) at {}'.format(self.name, id(self))
77013
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
202
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
203
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
204 class _DummyModuleLock:
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
205 """A simple _ModuleLock equivalent for Python builds without
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
206 multi-threading support."""
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
207
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
208 def __init__(self, name):
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
209 self.name = name
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
210 self.count = 0
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
211
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
212 def acquire(self):
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
213 self.count += 1
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
214 return True
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
215
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
216 def release(self):
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
217 if self.count == 0:
84582
33d379c2bb79 Issue #18415: Normalize what type of quotes are used with string
Brett Cannon <brett@python.org>
parents: 84468
diff changeset
218 raise RuntimeError('cannot release un-acquired lock')
77013
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
219 self.count -= 1
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
220
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
221 def __repr__(self):
84582
33d379c2bb79 Issue #18415: Normalize what type of quotes are used with string
Brett Cannon <brett@python.org>
parents: 84468
diff changeset
222 return '_DummyModuleLock({!r}) at {}'.format(self.name, id(self))
77013
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
223
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
224
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
225 # The following two functions are for consumption by Python/import.c.
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
226
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
227 def _get_module_lock(name):
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
228 """Get or create the module lock for a given module name.
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
229
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
230 Should only be called with the import lock taken."""
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
231 lock = None
78779
7fa6336e9864 Issue #15781: Fix two small race conditions in import's module locking.
Antoine Pitrou <solipsis@pitrou.net>
parents: 78758
diff changeset
232 try:
77013
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
233 lock = _module_locks[name]()
78779
7fa6336e9864 Issue #15781: Fix two small race conditions in import's module locking.
Antoine Pitrou <solipsis@pitrou.net>
parents: 78758
diff changeset
234 except KeyError:
7fa6336e9864 Issue #15781: Fix two small race conditions in import's module locking.
Antoine Pitrou <solipsis@pitrou.net>
parents: 78758
diff changeset
235 pass
77013
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
236 if lock is None:
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
237 if _thread is None:
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
238 lock = _DummyModuleLock(name)
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
239 else:
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
240 lock = _ModuleLock(name)
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
241 def cb(_):
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
242 del _module_locks[name]
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
243 _module_locks[name] = _weakref.ref(lock, cb)
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
244 return lock
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
245
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
246 def _lock_unlock_module(name):
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
247 """Release the global import lock, and acquires then release the
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
248 module lock for a given module name.
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
249 This is used to ensure a module is completely initialized, in the
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
250 event it is being imported by another thread.
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
251
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
252 Should only be called with the import lock taken."""
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
253 lock = _get_module_lock(name)
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
254 _imp.release_lock()
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
255 try:
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
256 lock.acquire()
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
257 except _DeadlockError:
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
258 # Concurrent circular import, we'll accept a partially initialized
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
259 # module object.
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
260 pass
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
261 else:
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
262 lock.release()
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
263
78344
62033490ca0f Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise
Nick Coghlan <ncoghlan@gmail.com>
parents: 78314
diff changeset
264 # Frame stripping magic ###############################################
77013
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
265
78344
62033490ca0f Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise
Nick Coghlan <ncoghlan@gmail.com>
parents: 78314
diff changeset
266 def _call_with_frames_removed(f, *args, **kwds):
62033490ca0f Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise
Nick Coghlan <ncoghlan@gmail.com>
parents: 78314
diff changeset
267 """remove_importlib_frames in import.c will always remove sequences
62033490ca0f Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise
Nick Coghlan <ncoghlan@gmail.com>
parents: 78314
diff changeset
268 of importlib frames that end with a call to this function
62033490ca0f Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise
Nick Coghlan <ncoghlan@gmail.com>
parents: 78314
diff changeset
269
62033490ca0f Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise
Nick Coghlan <ncoghlan@gmail.com>
parents: 78314
diff changeset
270 Use it instead of a normal call in places where including the importlib
62033490ca0f Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise
Nick Coghlan <ncoghlan@gmail.com>
parents: 78314
diff changeset
271 frames introduces unwanted noise into the traceback (e.g. when executing
62033490ca0f Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise
Nick Coghlan <ncoghlan@gmail.com>
parents: 78314
diff changeset
272 module code)
62033490ca0f Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise
Nick Coghlan <ncoghlan@gmail.com>
parents: 78314
diff changeset
273 """
62033490ca0f Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise
Nick Coghlan <ncoghlan@gmail.com>
parents: 78314
diff changeset
274 return f(*args, **kwds)
62033490ca0f Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise
Nick Coghlan <ncoghlan@gmail.com>
parents: 78314
diff changeset
275
62033490ca0f Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise
Nick Coghlan <ncoghlan@gmail.com>
parents: 78314
diff changeset
276
62033490ca0f Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise
Nick Coghlan <ncoghlan@gmail.com>
parents: 78314
diff changeset
277 # Finder/loader utility code ###############################################
52140
1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap.
Brett Cannon <bcannon@gmail.com>
parents: 52139
diff changeset
278
82957
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
279 # Magic word to reject .pyc files generated by other Python versions.
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
280 # It should change for each incompatible change to the bytecode.
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
281 #
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
282 # The value of CR and LF is incorporated so if you ever read or write
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
283 # a .pyc file in text mode the magic number will be wrong; also, the
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
284 # Apple MPW compiler swaps their values, botching string constants.
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
285 #
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
286 # The magic numbers must be spaced apart at least 2 values, as the
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
287 # -U interpeter flag will cause MAGIC+1 being used. They have been
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
288 # odd numbers for some time now.
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
289 #
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
290 # There were a variety of old schemes for setting the magic number.
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
291 # The current working scheme is to increment the previous value by
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
292 # 10.
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
293 #
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
294 # Starting with the adoption of PEP 3147 in Python 3.2, every bump in magic
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
295 # number also includes a new "magic tag", i.e. a human readable string used
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
296 # to represent the magic number in __pycache__ directories. When you change
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
297 # the magic number, you must also set a new unique magic tag. Generally this
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
298 # can be named after the Python major version of the magic number bump, but
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
299 # it can really be anything, as long as it's different than anything else
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
300 # that's come before. The tags are included in the following table, starting
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
301 # with Python 3.2a0.
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
302 #
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
303 # Known values:
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
304 # Python 1.5: 20121
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
305 # Python 1.5.1: 20121
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
306 # Python 1.5.2: 20121
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
307 # Python 1.6: 50428
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
308 # Python 2.0: 50823
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
309 # Python 2.0.1: 50823
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
310 # Python 2.1: 60202
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
311 # Python 2.1.1: 60202
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
312 # Python 2.1.2: 60202
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
313 # Python 2.2: 60717
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
314 # Python 2.3a0: 62011
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
315 # Python 2.3a0: 62021
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
316 # Python 2.3a0: 62011 (!)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
317 # Python 2.4a0: 62041
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
318 # Python 2.4a3: 62051
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
319 # Python 2.4b1: 62061
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
320 # Python 2.5a0: 62071
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
321 # Python 2.5a0: 62081 (ast-branch)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
322 # Python 2.5a0: 62091 (with)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
323 # Python 2.5a0: 62092 (changed WITH_CLEANUP opcode)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
324 # Python 2.5b3: 62101 (fix wrong code: for x, in ...)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
325 # Python 2.5b3: 62111 (fix wrong code: x += yield)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
326 # Python 2.5c1: 62121 (fix wrong lnotab with for loops and
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
327 # storing constants that should have been removed)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
328 # Python 2.5c2: 62131 (fix wrong code: for x, in ... in listcomp/genexp)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
329 # Python 2.6a0: 62151 (peephole optimizations and STORE_MAP opcode)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
330 # Python 2.6a1: 62161 (WITH_CLEANUP optimization)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
331 # Python 2.7a0: 62171 (optimize list comprehensions/change LIST_APPEND)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
332 # Python 2.7a0: 62181 (optimize conditional branches:
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
333 # introduce POP_JUMP_IF_FALSE and POP_JUMP_IF_TRUE)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
334 # Python 2.7a0 62191 (introduce SETUP_WITH)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
335 # Python 2.7a0 62201 (introduce BUILD_SET)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
336 # Python 2.7a0 62211 (introduce MAP_ADD and SET_ADD)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
337 # Python 3000: 3000
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
338 # 3010 (removed UNARY_CONVERT)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
339 # 3020 (added BUILD_SET)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
340 # 3030 (added keyword-only parameters)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
341 # 3040 (added signature annotations)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
342 # 3050 (print becomes a function)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
343 # 3060 (PEP 3115 metaclass syntax)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
344 # 3061 (string literals become unicode)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
345 # 3071 (PEP 3109 raise changes)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
346 # 3081 (PEP 3137 make __file__ and __name__ unicode)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
347 # 3091 (kill str8 interning)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
348 # 3101 (merge from 2.6a0, see 62151)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
349 # 3103 (__file__ points to source file)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
350 # Python 3.0a4: 3111 (WITH_CLEANUP optimization).
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
351 # Python 3.0a5: 3131 (lexical exception stacking, including POP_EXCEPT)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
352 # Python 3.1a0: 3141 (optimize list, set and dict comprehensions:
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
353 # change LIST_APPEND and SET_ADD, add MAP_ADD)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
354 # Python 3.1a0: 3151 (optimize conditional branches:
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
355 # introduce POP_JUMP_IF_FALSE and POP_JUMP_IF_TRUE)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
356 # Python 3.2a0: 3160 (add SETUP_WITH)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
357 # tag: cpython-32
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
358 # Python 3.2a1: 3170 (add DUP_TOP_TWO, remove DUP_TOPX and ROT_FOUR)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
359 # tag: cpython-32
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
360 # Python 3.2a2 3180 (add DELETE_DEREF)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
361 # Python 3.3a0 3190 __class__ super closure changed
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
362 # Python 3.3a0 3200 (__qualname__ added)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
363 # 3210 (added size modulo 2**32 to the pyc header)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
364 # Python 3.3a1 3220 (changed PEP 380 implementation)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
365 # Python 3.3a4 3230 (revert changes to implicit __class__ closure)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
366 # Python 3.4a1 3250 (evaluate positional default arguments before
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
367 # keyword-only defaults)
83572
cf65c7a75f55 check local class namespace before reaching for cells (closes #17853)
Benjamin Peterson <benjamin@python.org>
parents: 83376
diff changeset
368 # Python 3.4a1 3260 (add LOAD_CLASSDEREF; allow locals of class to override
cf65c7a75f55 check local class namespace before reaching for cells (closes #17853)
Benjamin Peterson <benjamin@python.org>
parents: 83376
diff changeset
369 # free vars)
84090
32527aa9ad16 typo fix
Brett Cannon <brett@python.org>
parents: 84046
diff changeset
370 # Python 3.4a1 3270 (various tweaks to the __class__ closure)
83791
6db3741e59be rather than passing locals to the class body, just execute the class body in the proper environment
Benjamin Peterson <benjamin@python.org>
parents: 83783
diff changeset
371 # Python 3.4a1 3280 (remove implicit class argument)
86491
35b384ed594b give explicitly global functions and classes a global __qualname__ (closes #19301)
Benjamin Peterson <benjamin@python.org>
parents: 86444
diff changeset
372 # Python 3.4a4 3290 (changes to __qualname__ computation)
86521
bb2affc1e317 cleanup the construction of __qualname__ (closes #19301 again)
Benjamin Peterson <benjamin@python.org>
parents: 86491
diff changeset
373 # Python 3.4a4 3300 (more changes to __qualname__ computation)
82957
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
374 #
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
375 # MAGIC must change whenever the bytecode emitted by the compiler may no
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
376 # longer be understood by older implementations of the eval loop (usually
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string
Victor Stinner <victor.stinner@gmail.com>
parents: 82876
diff changeset
377 # due to the addition of new opcodes).
78042
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in
Brett Cannon <brett@python.org>
parents: 78026
diff changeset
378
86521
bb2affc1e317 cleanup the construction of __qualname__ (closes #19301 again)
Benjamin Peterson <benjamin@python.org>
parents: 86491
diff changeset
379 MAGIC_NUMBER = (3300).to_bytes(2, 'little') + b'\r\n'
84128
5619bc2d8207 Issue #18192: Introduce importlib.util.MAGIC_NUMBER and document the
Brett Cannon <brett@python.org>
parents: 84109
diff changeset
380 _RAW_MAGIC_NUMBER = int.from_bytes(MAGIC_NUMBER, 'little') # For import.c
78042
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in
Brett Cannon <brett@python.org>
parents: 78026
diff changeset
381
76753
59b8a7fcb047 Properly mark names in importlib._bootstrap as private.
Brett Cannon <brett@python.org>
parents: 76586
diff changeset
382 _PYCACHE = '__pycache__'
76450
b773a751c2e7 Issue #13959: Re-implement imp.cache_from_source() in Lib/imp.py.
Brett Cannon <brett@python.org>
parents: 76449
diff changeset
383
76874
b81ddaf0db47 Issue #13959: Deprecate imp.get_suffixes() for new attributes on
Brett Cannon <brett@python.org>
parents: 76826
diff changeset
384 SOURCE_SUFFIXES = ['.py'] # _setup() adds .pyw as needed.
76451
ea46ebba8a0f Issue #13959: Re-implement imp.source_from_cache() in Lib/imp.py.
Brett Cannon <brett@python.org>
parents: 76450
diff changeset
385
76874
b81ddaf0db47 Issue #13959: Deprecate imp.get_suffixes() for new attributes on
Brett Cannon <brett@python.org>
parents: 76826
diff changeset
386 DEBUG_BYTECODE_SUFFIXES = ['.pyc']
b81ddaf0db47 Issue #13959: Deprecate imp.get_suffixes() for new attributes on
Brett Cannon <brett@python.org>
parents: 76826
diff changeset
387 OPTIMIZED_BYTECODE_SUFFIXES = ['.pyo']
76450
b773a751c2e7 Issue #13959: Re-implement imp.cache_from_source() in Lib/imp.py.
Brett Cannon <brett@python.org>
parents: 76449
diff changeset
388
76753
59b8a7fcb047 Properly mark names in importlib._bootstrap as private.
Brett Cannon <brett@python.org>
parents: 76586
diff changeset
389 def cache_from_source(path, debug_override=None):
76450
b773a751c2e7 Issue #13959: Re-implement imp.cache_from_source() in Lib/imp.py.
Brett Cannon <brett@python.org>
parents: 76449
diff changeset
390 """Given the path to a .py file, return the path to its .pyc/.pyo file.
b773a751c2e7 Issue #13959: Re-implement imp.cache_from_source() in Lib/imp.py.
Brett Cannon <brett@python.org>
parents: 76449
diff changeset
391
b773a751c2e7 Issue #13959: Re-implement imp.cache_from_source() in Lib/imp.py.
Brett Cannon <brett@python.org>
parents: 76449
diff changeset
392 The .py file does not need to exist; this simply returns the path to the
b773a751c2e7 Issue #13959: Re-implement imp.cache_from_source() in Lib/imp.py.
Brett Cannon <brett@python.org>
parents: 76449
diff changeset
393 .pyc/.pyo file calculated as if the .py file were imported. The extension
79173
4de5e4ec3cff don't depend on __debug__ because it's baked in at freeze time (issue #16046)
Benjamin Peterson <benjamin@python.org>
parents: 78779
diff changeset
394 will be .pyc unless sys.flags.optimize is non-zero, then it will be .pyo.
76450
b773a751c2e7 Issue #13959: Re-implement imp.cache_from_source() in Lib/imp.py.
Brett Cannon <brett@python.org>
parents: 76449
diff changeset
395
79191
9b0582477aad rephrase
Benjamin Peterson <benjamin@python.org>
parents: 79173
diff changeset
396 If debug_override is not None, then it must be a boolean and is used in
9b0582477aad rephrase
Benjamin Peterson <benjamin@python.org>
parents: 79173
diff changeset
397 place of sys.flags.optimize.
76450
b773a751c2e7 Issue #13959: Re-implement imp.cache_from_source() in Lib/imp.py.
Brett Cannon <brett@python.org>
parents: 76449
diff changeset
398
78026
e86330669bb5 Issue #15056: imp.cache_from_source() and source_from_cache() raise
Brett Cannon <brett@python.org>
parents: 78010
diff changeset
399 If sys.implementation.cache_tag is None then NotImplementedError is raised.
e86330669bb5 Issue #15056: imp.cache_from_source() and source_from_cache() raise
Brett Cannon <brett@python.org>
parents: 78010
diff changeset
400
76450
b773a751c2e7 Issue #13959: Re-implement imp.cache_from_source() in Lib/imp.py.
Brett Cannon <brett@python.org>
parents: 76449
diff changeset
401 """
79173
4de5e4ec3cff don't depend on __debug__ because it's baked in at freeze time (issue #16046)
Benjamin Peterson <benjamin@python.org>
parents: 78779
diff changeset
402 debug = not sys.flags.optimize if debug_override is None else debug_override
76874
b81ddaf0db47 Issue #13959: Deprecate imp.get_suffixes() for new attributes on
Brett Cannon <brett@python.org>
parents: 76826
diff changeset
403 if debug:
b81ddaf0db47 Issue #13959: Deprecate imp.get_suffixes() for new attributes on
Brett Cannon <brett@python.org>
parents: 76826
diff changeset
404 suffixes = DEBUG_BYTECODE_SUFFIXES
b81ddaf0db47 Issue #13959: Deprecate imp.get_suffixes() for new attributes on
Brett Cannon <brett@python.org>
parents: 76826
diff changeset
405 else:
b81ddaf0db47 Issue #13959: Deprecate imp.get_suffixes() for new attributes on
Brett Cannon <brett@python.org>
parents: 76826
diff changeset
406 suffixes = OPTIMIZED_BYTECODE_SUFFIXES
76450
b773a751c2e7 Issue #13959: Re-implement imp.cache_from_source() in Lib/imp.py.
Brett Cannon <brett@python.org>
parents: 76449
diff changeset
407 head, tail = _path_split(path)
b773a751c2e7 Issue #13959: Re-implement imp.cache_from_source() in Lib/imp.py.
Brett Cannon <brett@python.org>
parents: 76449
diff changeset
408 base_filename, sep, _ = tail.partition('.')
78026
e86330669bb5 Issue #15056: imp.cache_from_source() and source_from_cache() raise
Brett Cannon <brett@python.org>
parents: 78010
diff changeset
409 tag = sys.implementation.cache_tag
e86330669bb5 Issue #15056: imp.cache_from_source() and source_from_cache() raise
Brett Cannon <brett@python.org>
parents: 78010
diff changeset
410 if tag is None:
e86330669bb5 Issue #15056: imp.cache_from_source() and source_from_cache() raise
Brett Cannon <brett@python.org>
parents: 78010
diff changeset
411 raise NotImplementedError('sys.implementation.cache_tag is None')
e86330669bb5 Issue #15056: imp.cache_from_source() and source_from_cache() raise
Brett Cannon <brett@python.org>
parents: 78010
diff changeset
412 filename = ''.join([base_filename, sep, tag, suffixes[0]])
76753
59b8a7fcb047 Properly mark names in importlib._bootstrap as private.
Brett Cannon <brett@python.org>
parents: 76586
diff changeset
413 return _path_join(head, _PYCACHE, filename)
76450
b773a751c2e7 Issue #13959: Re-implement imp.cache_from_source() in Lib/imp.py.
Brett Cannon <brett@python.org>
parents: 76449
diff changeset
414
b773a751c2e7 Issue #13959: Re-implement imp.cache_from_source() in Lib/imp.py.
Brett Cannon <brett@python.org>
parents: 76449
diff changeset
415
78078
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use
Brett Cannon <brett@python.org>
parents: 78051
diff changeset
416 def source_from_cache(path):
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use
Brett Cannon <brett@python.org>
parents: 78051
diff changeset
417 """Given the path to a .pyc./.pyo file, return the path to its .py file.
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use
Brett Cannon <brett@python.org>
parents: 78051
diff changeset
418
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use
Brett Cannon <brett@python.org>
parents: 78051
diff changeset
419 The .pyc/.pyo file does not need to exist; this simply returns the path to
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use
Brett Cannon <brett@python.org>
parents: 78051
diff changeset
420 the .py file calculated to correspond to the .pyc/.pyo file. If path does
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use
Brett Cannon <brett@python.org>
parents: 78051
diff changeset
421 not conform to PEP 3147 format, ValueError will be raised. If
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use
Brett Cannon <brett@python.org>
parents: 78051
diff changeset
422 sys.implementation.cache_tag is None then NotImplementedError is raised.
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use
Brett Cannon <brett@python.org>
parents: 78051
diff changeset
423
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use
Brett Cannon <brett@python.org>
parents: 78051
diff changeset
424 """
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use
Brett Cannon <brett@python.org>
parents: 78051
diff changeset
425 if sys.implementation.cache_tag is None:
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use
Brett Cannon <brett@python.org>
parents: 78051
diff changeset
426 raise NotImplementedError('sys.implementation.cache_tag is None')
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use
Brett Cannon <brett@python.org>
parents: 78051
diff changeset
427 head, pycache_filename = _path_split(path)
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use
Brett Cannon <brett@python.org>
parents: 78051
diff changeset
428 head, pycache = _path_split(head)
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use
Brett Cannon <brett@python.org>
parents: 78051
diff changeset
429 if pycache != _PYCACHE:
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use
Brett Cannon <brett@python.org>
parents: 78051
diff changeset
430 raise ValueError('{} not bottom-level directory in '
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use
Brett Cannon <brett@python.org>
parents: 78051
diff changeset
431 '{!r}'.format(_PYCACHE, path))
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use
Brett Cannon <brett@python.org>
parents: 78051
diff changeset
432 if pycache_filename.count('.') != 2:
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use
Brett Cannon <brett@python.org>
parents: 78051
diff changeset
433 raise ValueError('expected only 2 dots in '
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use
Brett Cannon <brett@python.org>
parents: 78051
diff changeset
434 '{!r}'.format(pycache_filename))
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use
Brett Cannon <brett@python.org>
parents: 78051
diff changeset
435 base_filename = pycache_filename.partition('.')[0]
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use
Brett Cannon <brett@python.org>
parents: 78051
diff changeset
436 return _path_join(head, base_filename + SOURCE_SUFFIXES[0])
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use
Brett Cannon <brett@python.org>
parents: 78051
diff changeset
437
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use
Brett Cannon <brett@python.org>
parents: 78051
diff changeset
438
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use
Brett Cannon <brett@python.org>
parents: 78051
diff changeset
439 def _get_sourcefile(bytecode_path):
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use
Brett Cannon <brett@python.org>
parents: 78051
diff changeset
440 """Convert a bytecode file path to a source path (if possible).
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use
Brett Cannon <brett@python.org>
parents: 78051
diff changeset
441
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use
Brett Cannon <brett@python.org>
parents: 78051
diff changeset
442 This function exists purely for backwards-compatibility for
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use
Brett Cannon <brett@python.org>
parents: 78051
diff changeset
443 PyImport_ExecCodeModuleWithFilenames() in the C API.
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use
Brett Cannon <brett@python.org>
parents: 78051
diff changeset
444
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use
Brett Cannon <brett@python.org>
parents: 78051
diff changeset
445 """
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use
Brett Cannon <brett@python.org>
parents: 78051
diff changeset
446 if len(bytecode_path) == 0:
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use
Brett Cannon <brett@python.org>
parents: 78051
diff changeset
447 return None
84467
b8028f74bac4 Issue #18351: Fix various issues with
Brett Cannon <brett@python.org>
parents: 84174
diff changeset
448 rest, _, extension = bytecode_path.rpartition('.')
b8028f74bac4 Issue #18351: Fix various issues with
Brett Cannon <brett@python.org>
parents: 84174
diff changeset
449 if not rest or extension.lower()[-3:-1] != 'py':
78078
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use
Brett Cannon <brett@python.org>
parents: 78051
diff changeset
450 return bytecode_path
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use
Brett Cannon <brett@python.org>
parents: 78051
diff changeset
451 try:
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use
Brett Cannon <brett@python.org>
parents: 78051
diff changeset
452 source_path = source_from_cache(bytecode_path)
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use
Brett Cannon <brett@python.org>
parents: 78051
diff changeset
453 except (NotImplementedError, ValueError):
84467
b8028f74bac4 Issue #18351: Fix various issues with
Brett Cannon <brett@python.org>
parents: 84174
diff changeset
454 source_path = bytecode_path[:-1]
b8028f74bac4 Issue #18351: Fix various issues with
Brett Cannon <brett@python.org>
parents: 84174
diff changeset
455 return source_path if _path_isfile(source_path) else bytecode_path
78078
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use
Brett Cannon <brett@python.org>
parents: 78051
diff changeset
456
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use
Brett Cannon <brett@python.org>
parents: 78051
diff changeset
457
83376
04aaeae6ee7b Issue #17244: Don't mask exceptions raised during the creation of
Brett Cannon <brett@python.org>
parents: 83225
diff changeset
458 def _calc_mode(path):
04aaeae6ee7b Issue #17244: Don't mask exceptions raised during the creation of
Brett Cannon <brett@python.org>
parents: 83225
diff changeset
459 """Calculate the mode permissions for a bytecode file."""
04aaeae6ee7b Issue #17244: Don't mask exceptions raised during the creation of
Brett Cannon <brett@python.org>
parents: 83225
diff changeset
460 try:
04aaeae6ee7b Issue #17244: Don't mask exceptions raised during the creation of
Brett Cannon <brett@python.org>
parents: 83225
diff changeset
461 mode = _os.stat(path).st_mode
04aaeae6ee7b Issue #17244: Don't mask exceptions raised during the creation of
Brett Cannon <brett@python.org>
parents: 83225
diff changeset
462 except OSError:
04aaeae6ee7b Issue #17244: Don't mask exceptions raised during the creation of
Brett Cannon <brett@python.org>
parents: 83225
diff changeset
463 mode = 0o666
04aaeae6ee7b Issue #17244: Don't mask exceptions raised during the creation of
Brett Cannon <brett@python.org>
parents: 83225
diff changeset
464 # We always ensure write access so we can update cached files
04aaeae6ee7b Issue #17244: Don't mask exceptions raised during the creation of
Brett Cannon <brett@python.org>
parents: 83225
diff changeset
465 # later even when the source files are read-only on Windows (#6074)
04aaeae6ee7b Issue #17244: Don't mask exceptions raised during the creation of
Brett Cannon <brett@python.org>
parents: 83225
diff changeset
466 mode |= 0o200
04aaeae6ee7b Issue #17244: Don't mask exceptions raised during the creation of
Brett Cannon <brett@python.org>
parents: 83225
diff changeset
467 return mode
04aaeae6ee7b Issue #17244: Don't mask exceptions raised during the creation of
Brett Cannon <brett@python.org>
parents: 83225
diff changeset
468
04aaeae6ee7b Issue #17244: Don't mask exceptions raised during the creation of
Brett Cannon <brett@python.org>
parents: 83225
diff changeset
469
83055
ac89e7bc0e2f Issue #17357: Add missing verbosity messages when running under
Brett Cannon <brett@python.org>
parents: 82236
diff changeset
470 def _verbose_message(message, *args, verbosity=1):
76303
2dd046be2c88 Issue #2377: Make importlib the implementation of __import__().
Brett Cannon <brett@python.org>
parents: 76280
diff changeset
471 """Print the message to stderr if -v/PYTHONVERBOSE is turned on."""
83055
ac89e7bc0e2f Issue #17357: Add missing verbosity messages when running under
Brett Cannon <brett@python.org>
parents: 82236
diff changeset
472 if sys.flags.verbose >= verbosity:
76327
1ca10eb58483 utilize startswith(tuple)
Philip Jenvey <pjenvey@underboss.org>
parents: 76312
diff changeset
473 if not message.startswith(('#', 'import ')):
76303
2dd046be2c88 Issue #2377: Make importlib the implementation of __import__().
Brett Cannon <brett@python.org>
parents: 76280
diff changeset
474 message = '# ' + message
2dd046be2c88 Issue #2377: Make importlib the implementation of __import__().
Brett Cannon <brett@python.org>
parents: 76280
diff changeset
475 print(message.format(*args), file=sys.stderr)
2dd046be2c88 Issue #2377: Make importlib the implementation of __import__().
Brett Cannon <brett@python.org>
parents: 76280
diff changeset
476
52140
1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap.
Brett Cannon <bcannon@gmail.com>
parents: 52139
diff changeset
477
84171
205aa49e5cd5 Issue #18115: Abstract out managing the cleanup of modules to use in
Brett Cannon <brett@python.org>
parents: 84169
diff changeset
478 class _ManageReload:
205aa49e5cd5 Issue #18115: Abstract out managing the cleanup of modules to use in
Brett Cannon <brett@python.org>
parents: 84169
diff changeset
479
205aa49e5cd5 Issue #18115: Abstract out managing the cleanup of modules to use in
Brett Cannon <brett@python.org>
parents: 84169
diff changeset
480 def __init__(self, name):
205aa49e5cd5 Issue #18115: Abstract out managing the cleanup of modules to use in
Brett Cannon <brett@python.org>
parents: 84169
diff changeset
481 self._name = name
205aa49e5cd5 Issue #18115: Abstract out managing the cleanup of modules to use in
Brett Cannon <brett@python.org>
parents: 84169
diff changeset
482
205aa49e5cd5 Issue #18115: Abstract out managing the cleanup of modules to use in
Brett Cannon <brett@python.org>
parents: 84169
diff changeset
483 def __enter__(self):
205aa49e5cd5 Issue #18115: Abstract out managing the cleanup of modules to use in
Brett Cannon <brett@python.org>
parents: 84169
diff changeset
484 self._is_reload = self._name in sys.modules
205aa49e5cd5 Issue #18115: Abstract out managing the cleanup of modules to use in
Brett Cannon <brett@python.org>
parents: 84169
diff changeset
485
205aa49e5cd5 Issue #18115: Abstract out managing the cleanup of modules to use in
Brett Cannon <brett@python.org>
parents: 84169
diff changeset
486 def __exit__(self, *args):
205aa49e5cd5 Issue #18115: Abstract out managing the cleanup of modules to use in
Brett Cannon <brett@python.org>
parents: 84169
diff changeset
487 if any(arg is not None for arg in args) and not self._is_reload:
205aa49e5cd5 Issue #18115: Abstract out managing the cleanup of modules to use in
Brett Cannon <brett@python.org>
parents: 84169
diff changeset
488 try:
205aa49e5cd5 Issue #18115: Abstract out managing the cleanup of modules to use in
Brett Cannon <brett@python.org>
parents: 84169
diff changeset
489 del sys.modules[self._name]
205aa49e5cd5 Issue #18115: Abstract out managing the cleanup of modules to use in
Brett Cannon <brett@python.org>
parents: 84169
diff changeset
490 except KeyError:
205aa49e5cd5 Issue #18115: Abstract out managing the cleanup of modules to use in
Brett Cannon <brett@python.org>
parents: 84169
diff changeset
491 pass
205aa49e5cd5 Issue #18115: Abstract out managing the cleanup of modules to use in
Brett Cannon <brett@python.org>
parents: 84169
diff changeset
492
205aa49e5cd5 Issue #18115: Abstract out managing the cleanup of modules to use in
Brett Cannon <brett@python.org>
parents: 84169
diff changeset
493
83988
6ace4133c1c0 Rename importlib.util.ModuleManager to module_to_load so that the name
Brett Cannon <brett@python.org>
parents: 83963
diff changeset
494 # Written as a class only because contextlib is not available.
84171
205aa49e5cd5 Issue #18115: Abstract out managing the cleanup of modules to use in
Brett Cannon <brett@python.org>
parents: 84169
diff changeset
495 class _ModuleManager(_ManageReload):
83960
3a1976680717 Introduce importlib.util.ModuleManager which is a context manager to
Brett Cannon <brett@python.org>
parents: 83791
diff changeset
496
3a1976680717 Introduce importlib.util.ModuleManager which is a context manager to
Brett Cannon <brett@python.org>
parents: 83791
diff changeset
497 """Context manager which returns the module to be loaded.
3a1976680717 Introduce importlib.util.ModuleManager which is a context manager to
Brett Cannon <brett@python.org>
parents: 83791
diff changeset
498
3a1976680717 Introduce importlib.util.ModuleManager which is a context manager to
Brett Cannon <brett@python.org>
parents: 83791
diff changeset
499 Does the proper unloading from sys.modules upon failure.
3a1976680717 Introduce importlib.util.ModuleManager which is a context manager to
Brett Cannon <brett@python.org>
parents: 83791
diff changeset
500
3a1976680717 Introduce importlib.util.ModuleManager which is a context manager to
Brett Cannon <brett@python.org>
parents: 83791
diff changeset
501 """
3a1976680717 Introduce importlib.util.ModuleManager which is a context manager to
Brett Cannon <brett@python.org>
parents: 83791
diff changeset
502
83998
39cc1b04713e Add a reset_name argument to importlib.util.module_to_load in order to
Brett Cannon <brett@python.org>
parents: 83988
diff changeset
503 def __init__(self, name, *, reset_name=True):
39cc1b04713e Add a reset_name argument to importlib.util.module_to_load in order to
Brett Cannon <brett@python.org>
parents: 83988
diff changeset
504 """Prepare the context manager.
39cc1b04713e Add a reset_name argument to importlib.util.module_to_load in order to
Brett Cannon <brett@python.org>
parents: 83988
diff changeset
505
39cc1b04713e Add a reset_name argument to importlib.util.module_to_load in order to
Brett Cannon <brett@python.org>
parents: 83988
diff changeset
506 The reset_name argument specifies whether to unconditionally reset
39cc1b04713e Add a reset_name argument to importlib.util.module_to_load in order to
Brett Cannon <brett@python.org>
parents: 83988
diff changeset
507 the __name__ attribute if the module is found to be a reload.
39cc1b04713e Add a reset_name argument to importlib.util.module_to_load in order to
Brett Cannon <brett@python.org>
parents: 83988
diff changeset
508 """
84171
205aa49e5cd5 Issue #18115: Abstract out managing the cleanup of modules to use in
Brett Cannon <brett@python.org>
parents: 84169
diff changeset
509 super().__init__(name)
83998
39cc1b04713e Add a reset_name argument to importlib.util.module_to_load in order to
Brett Cannon <brett@python.org>
parents: 83988
diff changeset
510 self._reset_name = reset_name
83960
3a1976680717 Introduce importlib.util.ModuleManager which is a context manager to
Brett Cannon <brett@python.org>
parents: 83791
diff changeset
511
3a1976680717 Introduce importlib.util.ModuleManager which is a context manager to
Brett Cannon <brett@python.org>
parents: 83791
diff changeset
512 def __enter__(self):
84171
205aa49e5cd5 Issue #18115: Abstract out managing the cleanup of modules to use in
Brett Cannon <brett@python.org>
parents: 84169
diff changeset
513 super().__enter__()
83960
3a1976680717 Introduce importlib.util.ModuleManager which is a context manager to
Brett Cannon <brett@python.org>
parents: 83791
diff changeset
514 self._module = sys.modules.get(self._name)
3a1976680717 Introduce importlib.util.ModuleManager which is a context manager to
Brett Cannon <brett@python.org>
parents: 83791
diff changeset
515 if not self._is_reload:
3a1976680717 Introduce importlib.util.ModuleManager which is a context manager to
Brett Cannon <brett@python.org>
parents: 83791
diff changeset
516 # This must be done before open() is called as the 'io' module
3a1976680717 Introduce importlib.util.ModuleManager which is a context manager to
Brett Cannon <brett@python.org>
parents: 83791
diff changeset
517 # implicitly imports 'locale' and would otherwise trigger an
3a1976680717 Introduce importlib.util.ModuleManager which is a context manager to
Brett Cannon <brett@python.org>
parents: 83791
diff changeset
518 # infinite loop.
84134
9cacdb9d0c59 Issue #17907: touch up the code for imp.new_module().
Brett Cannon <brett@python.org>
parents: 84128
diff changeset
519 self._module = type(_io)(self._name)
83960
3a1976680717 Introduce importlib.util.ModuleManager which is a context manager to
Brett Cannon <brett@python.org>
parents: 83791
diff changeset
520 # This must be done before putting the module in sys.modules
3a1976680717 Introduce importlib.util.ModuleManager which is a context manager to
Brett Cannon <brett@python.org>
parents: 83791
diff changeset
521 # (otherwise an optimization shortcut in import.c becomes wrong)
3a1976680717 Introduce importlib.util.ModuleManager which is a context manager to
Brett Cannon <brett@python.org>
parents: 83791
diff changeset
522 self._module.__initializing__ = True
3a1976680717 Introduce importlib.util.ModuleManager which is a context manager to
Brett Cannon <brett@python.org>
parents: 83791
diff changeset
523 sys.modules[self._name] = self._module
83998
39cc1b04713e Add a reset_name argument to importlib.util.module_to_load in order to
Brett Cannon <brett@python.org>
parents: 83988
diff changeset
524 elif self._reset_name:
39cc1b04713e Add a reset_name argument to importlib.util.module_to_load in order to
Brett Cannon <brett@python.org>
parents: 83988
diff changeset
525 try:
39cc1b04713e Add a reset_name argument to importlib.util.module_to_load in order to
Brett Cannon <brett@python.org>
parents: 83988
diff changeset
526 self._module.__name__ = self._name
39cc1b04713e Add a reset_name argument to importlib.util.module_to_load in order to
Brett Cannon <brett@python.org>
parents: 83988
diff changeset
527 except AttributeError:
39cc1b04713e Add a reset_name argument to importlib.util.module_to_load in order to
Brett Cannon <brett@python.org>
parents: 83988
diff changeset
528 pass
83960
3a1976680717 Introduce importlib.util.ModuleManager which is a context manager to
Brett Cannon <brett@python.org>
parents: 83791
diff changeset
529 return self._module
3a1976680717 Introduce importlib.util.ModuleManager which is a context manager to
Brett Cannon <brett@python.org>
parents: 83791
diff changeset
530
3a1976680717 Introduce importlib.util.ModuleManager which is a context manager to
Brett Cannon <brett@python.org>
parents: 83791
diff changeset
531 def __exit__(self, *args):
3a1976680717 Introduce importlib.util.ModuleManager which is a context manager to
Brett Cannon <brett@python.org>
parents: 83791
diff changeset
532 self._module.__initializing__ = False
3a1976680717 Introduce importlib.util.ModuleManager which is a context manager to
Brett Cannon <brett@python.org>
parents: 83791
diff changeset
533 del self._module
84171
205aa49e5cd5 Issue #18115: Abstract out managing the cleanup of modules to use in
Brett Cannon <brett@python.org>
parents: 84169
diff changeset
534 super().__exit__(*args)
83960
3a1976680717 Introduce importlib.util.ModuleManager which is a context manager to
Brett Cannon <brett@python.org>
parents: 83791
diff changeset
535
3a1976680717 Introduce importlib.util.ModuleManager which is a context manager to
Brett Cannon <brett@python.org>
parents: 83791
diff changeset
536
83999
130dfd6b3428 Fix for last commit on adding reset_name to module_to_load
Brett Cannon <brett@python.org>
parents: 83998
diff changeset
537 def module_to_load(name, *, reset_name=True):
130dfd6b3428 Fix for last commit on adding reset_name to module_to_load
Brett Cannon <brett@python.org>
parents: 83998
diff changeset
538 """Return a context manager which provides the module object to load.
130dfd6b3428 Fix for last commit on adding reset_name to module_to_load
Brett Cannon <brett@python.org>
parents: 83998
diff changeset
539
130dfd6b3428 Fix for last commit on adding reset_name to module_to_load
Brett Cannon <brett@python.org>
parents: 83998
diff changeset
540 If reset_name is true, reset the module's __name__ to 'name'.
130dfd6b3428 Fix for last commit on adding reset_name to module_to_load
Brett Cannon <brett@python.org>
parents: 83998
diff changeset
541 """
83988
6ace4133c1c0 Rename importlib.util.ModuleManager to module_to_load so that the name
Brett Cannon <brett@python.org>
parents: 83963
diff changeset
542 # Hiding _ModuleManager behind a function for better naming.
83999
130dfd6b3428 Fix for last commit on adding reset_name to module_to_load
Brett Cannon <brett@python.org>
parents: 83998
diff changeset
543 return _ModuleManager(name, reset_name=reset_name)
83988
6ace4133c1c0 Rename importlib.util.ModuleManager to module_to_load so that the name
Brett Cannon <brett@python.org>
parents: 83963
diff changeset
544
6ace4133c1c0 Rename importlib.util.ModuleManager to module_to_load so that the name
Brett Cannon <brett@python.org>
parents: 83963
diff changeset
545
84001
e873f2e67353 Issues #18088, 18089: Introduce
Brett Cannon <brett@python.org>
parents: 83999
diff changeset
546 def _init_package_attrs(loader, module):
e873f2e67353 Issues #18088, 18089: Introduce
Brett Cannon <brett@python.org>
parents: 83999
diff changeset
547 """Set __package__ and __path__ based on what loader.is_package() says."""
e873f2e67353 Issues #18088, 18089: Introduce
Brett Cannon <brett@python.org>
parents: 83999
diff changeset
548 name = module.__name__
e873f2e67353 Issues #18088, 18089: Introduce
Brett Cannon <brett@python.org>
parents: 83999
diff changeset
549 try:
e873f2e67353 Issues #18088, 18089: Introduce
Brett Cannon <brett@python.org>
parents: 83999
diff changeset
550 is_package = loader.is_package(name)
e873f2e67353 Issues #18088, 18089: Introduce
Brett Cannon <brett@python.org>
parents: 83999
diff changeset
551 except ImportError:
e873f2e67353 Issues #18088, 18089: Introduce
Brett Cannon <brett@python.org>
parents: 83999
diff changeset
552 pass
e873f2e67353 Issues #18088, 18089: Introduce
Brett Cannon <brett@python.org>
parents: 83999
diff changeset
553 else:
e873f2e67353 Issues #18088, 18089: Introduce
Brett Cannon <brett@python.org>
parents: 83999
diff changeset
554 if is_package:
e873f2e67353 Issues #18088, 18089: Introduce
Brett Cannon <brett@python.org>
parents: 83999
diff changeset
555 module.__package__ = name
e873f2e67353 Issues #18088, 18089: Introduce
Brett Cannon <brett@python.org>
parents: 83999
diff changeset
556 module.__path__ = []
e873f2e67353 Issues #18088, 18089: Introduce
Brett Cannon <brett@python.org>
parents: 83999
diff changeset
557 else:
e873f2e67353 Issues #18088, 18089: Introduce
Brett Cannon <brett@python.org>
parents: 83999
diff changeset
558 module.__package__ = name.rpartition('.')[0]
e873f2e67353 Issues #18088, 18089: Introduce
Brett Cannon <brett@python.org>
parents: 83999
diff changeset
559
e873f2e67353 Issues #18088, 18089: Introduce
Brett Cannon <brett@python.org>
parents: 83999
diff changeset
560
e873f2e67353 Issues #18088, 18089: Introduce
Brett Cannon <brett@python.org>
parents: 83999
diff changeset
561 def _init_file_attrs(loader, module):
e873f2e67353 Issues #18088, 18089: Introduce
Brett Cannon <brett@python.org>
parents: 83999
diff changeset
562 """Set __file__ and __path__ based on loader.get_filename()."""
e873f2e67353 Issues #18088, 18089: Introduce
Brett Cannon <brett@python.org>
parents: 83999
diff changeset
563 try:
e873f2e67353 Issues #18088, 18089: Introduce
Brett Cannon <brett@python.org>
parents: 83999
diff changeset
564 module.__file__ = loader.get_filename(module.__name__)
e873f2e67353 Issues #18088, 18089: Introduce
Brett Cannon <brett@python.org>
parents: 83999
diff changeset
565 except ImportError:
e873f2e67353 Issues #18088, 18089: Introduce
Brett Cannon <brett@python.org>
parents: 83999
diff changeset
566 pass
e873f2e67353 Issues #18088, 18089: Introduce
Brett Cannon <brett@python.org>
parents: 83999
diff changeset
567 else:
e873f2e67353 Issues #18088, 18089: Introduce
Brett Cannon <brett@python.org>
parents: 83999
diff changeset
568 if module.__name__ == module.__package__:
e873f2e67353 Issues #18088, 18089: Introduce
Brett Cannon <brett@python.org>
parents: 83999
diff changeset
569 module.__path__.append(_path_split(module.__file__)[0])
e873f2e67353 Issues #18088, 18089: Introduce
Brett Cannon <brett@python.org>
parents: 83999
diff changeset
570
e873f2e67353 Issues #18088, 18089: Introduce
Brett Cannon <brett@python.org>
parents: 83999
diff changeset
571
52002
76a579b6bade Rename importlib.util.set___package__ to set_package.
Brett Cannon <bcannon@gmail.com>
parents: 51995
diff changeset
572 def set_package(fxn):
51430
5cdc5f0c134c Move importlib completely over to using rpartition and accepting the empty
Brett Cannon <bcannon@gmail.com>
parents: 51429
diff changeset
573 """Set __package__ on the returned module."""
74449
9935f7837c4b Issue #13588: Rename decorators in importlib.
Brett Cannon <brett@python.org>
parents: 74358
diff changeset
574 def set_package_wrapper(*args, **kwargs):
51430
5cdc5f0c134c Move importlib completely over to using rpartition and accepting the empty
Brett Cannon <bcannon@gmail.com>
parents: 51429
diff changeset
575 module = fxn(*args, **kwargs)
77964
37850f083af5 Minor refactoring in importlib._bootstrap, and fix the '_wrap' docstring.
Florent Xicluna <florent.xicluna@gmail.com>
parents: 77917
diff changeset
576 if getattr(module, '__package__', None) is None:
51430
5cdc5f0c134c Move importlib completely over to using rpartition and accepting the empty
Brett Cannon <bcannon@gmail.com>
parents: 51429
diff changeset
577 module.__package__ = module.__name__
5cdc5f0c134c Move importlib completely over to using rpartition and accepting the empty
Brett Cannon <bcannon@gmail.com>
parents: 51429
diff changeset
578 if not hasattr(module, '__path__'):
5cdc5f0c134c Move importlib completely over to using rpartition and accepting the empty
Brett Cannon <bcannon@gmail.com>
parents: 51429
diff changeset
579 module.__package__ = module.__package__.rpartition('.')[0]
5cdc5f0c134c Move importlib completely over to using rpartition and accepting the empty
Brett Cannon <bcannon@gmail.com>
parents: 51429
diff changeset
580 return module
74449
9935f7837c4b Issue #13588: Rename decorators in importlib.
Brett Cannon <brett@python.org>
parents: 74358
diff changeset
581 _wrap(set_package_wrapper, fxn)
9935f7837c4b Issue #13588: Rename decorators in importlib.
Brett Cannon <brett@python.org>
parents: 74358
diff changeset
582 return set_package_wrapper
51430
5cdc5f0c134c Move importlib completely over to using rpartition and accepting the empty
Brett Cannon <bcannon@gmail.com>
parents: 51429
diff changeset
583
5cdc5f0c134c Move importlib completely over to using rpartition and accepting the empty
Brett Cannon <bcannon@gmail.com>
parents: 51429
diff changeset
584
52115
afd7118c8ccf Implement importlib.util.set_loader: a decorator to automatically set
Brett Cannon <bcannon@gmail.com>
parents: 52112
diff changeset
585 def set_loader(fxn):
afd7118c8ccf Implement importlib.util.set_loader: a decorator to automatically set
Brett Cannon <bcannon@gmail.com>
parents: 52112
diff changeset
586 """Set __loader__ on the returned module."""
74449
9935f7837c4b Issue #13588: Rename decorators in importlib.
Brett Cannon <brett@python.org>
parents: 74358
diff changeset
587 def set_loader_wrapper(self, *args, **kwargs):
52115
afd7118c8ccf Implement importlib.util.set_loader: a decorator to automatically set
Brett Cannon <bcannon@gmail.com>
parents: 52112
diff changeset
588 module = fxn(self, *args, **kwargs)
82643
7647aae9481c Issue #17117: Have both import itself and importlib.util.set_loader()
Brett Cannon <brett@python.org>
parents: 82385
diff changeset
589 if getattr(module, '__loader__', None) is None:
52115
afd7118c8ccf Implement importlib.util.set_loader: a decorator to automatically set
Brett Cannon <bcannon@gmail.com>
parents: 52112
diff changeset
590 module.__loader__ = self
afd7118c8ccf Implement importlib.util.set_loader: a decorator to automatically set
Brett Cannon <bcannon@gmail.com>
parents: 52112
diff changeset
591 return module
74449
9935f7837c4b Issue #13588: Rename decorators in importlib.
Brett Cannon <brett@python.org>
parents: 74358
diff changeset
592 _wrap(set_loader_wrapper, fxn)
9935f7837c4b Issue #13588: Rename decorators in importlib.
Brett Cannon <brett@python.org>
parents: 74358
diff changeset
593 return set_loader_wrapper
52115
afd7118c8ccf Implement importlib.util.set_loader: a decorator to automatically set
Brett Cannon <bcannon@gmail.com>
parents: 52112
diff changeset
594
afd7118c8ccf Implement importlib.util.set_loader: a decorator to automatically set
Brett Cannon <bcannon@gmail.com>
parents: 52112
diff changeset
595
52140
1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap.
Brett Cannon <bcannon@gmail.com>
parents: 52139
diff changeset
596 def _check_name(method):
1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap.
Brett Cannon <bcannon@gmail.com>
parents: 52139
diff changeset
597 """Decorator to verify that the module being requested matches the one the
1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap.
Brett Cannon <bcannon@gmail.com>
parents: 52139
diff changeset
598 loader can handle.
1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap.
Brett Cannon <bcannon@gmail.com>
parents: 52139
diff changeset
599
1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap.
Brett Cannon <bcannon@gmail.com>
parents: 52139
diff changeset
600 The first argument (self) must define _name which the second argument is
68488
0991b40e895d #11515: fix several typos. Patch by Piotr Kasprzyk.
Ezio Melotti <ezio.melotti@gmail.com>
parents: 58960
diff changeset
601 compared against. If the comparison fails then ImportError is raised.
52140
1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap.
Brett Cannon <bcannon@gmail.com>
parents: 52139
diff changeset
602
1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap.
Brett Cannon <bcannon@gmail.com>
parents: 52139
diff changeset
603 """
76877
626d5c6fbd95 Issue #13959: Have
Brett Cannon <brett@python.org>
parents: 76874
diff changeset
604 def _check_name_wrapper(self, name=None, *args, **kwargs):
626d5c6fbd95 Issue #13959: Have
Brett Cannon <brett@python.org>
parents: 76874
diff changeset
605 if name is None:
626d5c6fbd95 Issue #13959: Have
Brett Cannon <brett@python.org>
parents: 76874
diff changeset
606 name = self.name
626d5c6fbd95 Issue #13959: Have
Brett Cannon <brett@python.org>
parents: 76874
diff changeset
607 elif self.name != name:
84582
33d379c2bb79 Issue #18415: Normalize what type of quotes are used with string
Brett Cannon <brett@python.org>
parents: 84468
diff changeset
608 raise ImportError('loader cannot handle %s' % name, name=name)
52140
1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap.
Brett Cannon <bcannon@gmail.com>
parents: 52139
diff changeset
609 return method(self, name, *args, **kwargs)
74449
9935f7837c4b Issue #13588: Rename decorators in importlib.
Brett Cannon <brett@python.org>
parents: 74358
diff changeset
610 _wrap(_check_name_wrapper, method)
9935f7837c4b Issue #13588: Rename decorators in importlib.
Brett Cannon <brett@python.org>
parents: 74358
diff changeset
611 return _check_name_wrapper
52140
1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap.
Brett Cannon <bcannon@gmail.com>
parents: 52139
diff changeset
612
1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap.
Brett Cannon <bcannon@gmail.com>
parents: 52139
diff changeset
613
52166
69cd615b76fb Implement InspectLoader for BuiltinImporter.
Brett Cannon <bcannon@gmail.com>
parents: 52165
diff changeset
614 def _requires_builtin(fxn):
69cd615b76fb Implement InspectLoader for BuiltinImporter.
Brett Cannon <bcannon@gmail.com>
parents: 52165
diff changeset
615 """Decorator to verify the named module is built-in."""
74449
9935f7837c4b Issue #13588: Rename decorators in importlib.
Brett Cannon <brett@python.org>
parents: 74358
diff changeset
616 def _requires_builtin_wrapper(self, fullname):
52166
69cd615b76fb Implement InspectLoader for BuiltinImporter.
Brett Cannon <bcannon@gmail.com>
parents: 52165
diff changeset
617 if fullname not in sys.builtin_module_names:
84582
33d379c2bb79 Issue #18415: Normalize what type of quotes are used with string
Brett Cannon <brett@python.org>
parents: 84468
diff changeset
618 raise ImportError('{} is not a built-in module'.format(fullname),
76280
c071af95772a Have importlib take advantage of ImportError's new 'name' and 'path'
Brett Cannon <brett@python.org>
parents: 76081
diff changeset
619 name=fullname)
52166
69cd615b76fb Implement InspectLoader for BuiltinImporter.
Brett Cannon <bcannon@gmail.com>
parents: 52165
diff changeset
620 return fxn(self, fullname)
74449
9935f7837c4b Issue #13588: Rename decorators in importlib.
Brett Cannon <brett@python.org>
parents: 74358
diff changeset
621 _wrap(_requires_builtin_wrapper, fxn)
9935f7837c4b Issue #13588: Rename decorators in importlib.
Brett Cannon <brett@python.org>
parents: 74358
diff changeset
622 return _requires_builtin_wrapper
52166
69cd615b76fb Implement InspectLoader for BuiltinImporter.
Brett Cannon <bcannon@gmail.com>
parents: 52165
diff changeset
623
69cd615b76fb Implement InspectLoader for BuiltinImporter.
Brett Cannon <bcannon@gmail.com>
parents: 52165
diff changeset
624
52168
70f5f9113ae1 Implement InspectLoader for FrozenImporter.
Brett Cannon <bcannon@gmail.com>
parents: 52166
diff changeset
625 def _requires_frozen(fxn):
70f5f9113ae1 Implement InspectLoader for FrozenImporter.
Brett Cannon <bcannon@gmail.com>
parents: 52166
diff changeset
626 """Decorator to verify the named module is frozen."""
74449
9935f7837c4b Issue #13588: Rename decorators in importlib.
Brett Cannon <brett@python.org>
parents: 74358
diff changeset
627 def _requires_frozen_wrapper(self, fullname):
76331
d777f854a66e Issue #13959: Rename imp to _imp and add Lib/imp.py and begin
Brett Cannon <brett@python.org>
parents: 76330
diff changeset
628 if not _imp.is_frozen(fullname):
84582
33d379c2bb79 Issue #18415: Normalize what type of quotes are used with string
Brett Cannon <brett@python.org>
parents: 84468
diff changeset
629 raise ImportError('{} is not a frozen module'.format(fullname),
76280
c071af95772a Have importlib take advantage of ImportError's new 'name' and 'path'
Brett Cannon <brett@python.org>
parents: 76081
diff changeset
630 name=fullname)
52168
70f5f9113ae1 Implement InspectLoader for FrozenImporter.
Brett Cannon <bcannon@gmail.com>
parents: 52166
diff changeset
631 return fxn(self, fullname)
74449
9935f7837c4b Issue #13588: Rename decorators in importlib.
Brett Cannon <brett@python.org>
parents: 74358
diff changeset
632 _wrap(_requires_frozen_wrapper, fxn)
9935f7837c4b Issue #13588: Rename decorators in importlib.
Brett Cannon <brett@python.org>
parents: 74358
diff changeset
633 return _requires_frozen_wrapper
52168
70f5f9113ae1 Implement InspectLoader for FrozenImporter.
Brett Cannon <bcannon@gmail.com>
parents: 52166
diff changeset
634
70f5f9113ae1 Implement InspectLoader for FrozenImporter.
Brett Cannon <bcannon@gmail.com>
parents: 52166
diff changeset
635
78489
e7a67f1bf604 Issue #15502: Refactor some code.
Brett Cannon <brett@python.org>
parents: 78487
diff changeset
636 def _find_module_shim(self, fullname):
e7a67f1bf604 Issue #15502: Refactor some code.
Brett Cannon <brett@python.org>
parents: 78487
diff changeset
637 """Try to find a loader for the specified module by delegating to
e7a67f1bf604 Issue #15502: Refactor some code.
Brett Cannon <brett@python.org>
parents: 78487
diff changeset
638 self.find_loader()."""
e7a67f1bf604 Issue #15502: Refactor some code.
Brett Cannon <brett@python.org>
parents: 78487
diff changeset
639 # Call find_loader(). If it returns a string (indicating this
e7a67f1bf604 Issue #15502: Refactor some code.
Brett Cannon <brett@python.org>
parents: 78487
diff changeset
640 # is a namespace package portion), generate a warning and
e7a67f1bf604 Issue #15502: Refactor some code.
Brett Cannon <brett@python.org>
parents: 78487
diff changeset
641 # return None.
e7a67f1bf604 Issue #15502: Refactor some code.
Brett Cannon <brett@python.org>
parents: 78487
diff changeset
642 loader, portions = self.find_loader(fullname)
e7a67f1bf604 Issue #15502: Refactor some code.
Brett Cannon <brett@python.org>
parents: 78487
diff changeset
643 if loader is None and len(portions):
84582
33d379c2bb79 Issue #18415: Normalize what type of quotes are used with string
Brett Cannon <brett@python.org>
parents: 84468
diff changeset
644 msg = 'Not importing directory {}: missing __init__'
78489
e7a67f1bf604 Issue #15502: Refactor some code.
Brett Cannon <brett@python.org>
parents: 78487
diff changeset
645 _warnings.warn(msg.format(portions[0]), ImportWarning)
e7a67f1bf604 Issue #15502: Refactor some code.
Brett Cannon <brett@python.org>
parents: 78487
diff changeset
646 return loader
e7a67f1bf604 Issue #15502: Refactor some code.
Brett Cannon <brett@python.org>
parents: 78487
diff changeset
647
e7a67f1bf604 Issue #15502: Refactor some code.
Brett Cannon <brett@python.org>
parents: 78487
diff changeset
648
81432
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
649 def _validate_bytecode_header(data, source_stats=None, name=None, path=None):
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
650 """Validate the header of the passed-in bytecode against source_stats (if
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
651 given) and returning the bytecode that can be compiled by compile().
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
652
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
653 All other arguments are used to enhance error reporting.
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
654
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
655 ImportError is raised when the magic number is incorrect or the bytecode is
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
656 found to be stale. EOFError is raised when the data is found to be
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
657 truncated.
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
658
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
659 """
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
660 exc_details = {}
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
661 if name is not None:
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
662 exc_details['name'] = name
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
663 else:
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
664 # To prevent having to make all messages have a conditional name.
81735
792810303239 Touch up exception messaging
Brett Cannon <brett@python.org>
parents: 81432
diff changeset
665 name = '<bytecode>'
81432
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
666 if path is not None:
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
667 exc_details['path'] = path
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
668 magic = data[:4]
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
669 raw_timestamp = data[4:8]
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
670 raw_size = data[8:12]
84128
5619bc2d8207 Issue #18192: Introduce importlib.util.MAGIC_NUMBER and document the
Brett Cannon <brett@python.org>
parents: 84109
diff changeset
671 if magic != MAGIC_NUMBER:
83056
Brett Cannon <brett@python.org>
parents: 82957 83055
diff changeset
672 message = 'bad magic number in {!r}: {!r}'.format(name, magic)
Brett Cannon <brett@python.org>
parents: 82957 83055
diff changeset
673 _verbose_message(message)
Brett Cannon <brett@python.org>
parents: 82957 83055
diff changeset
674 raise ImportError(message, **exc_details)
81432
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
675 elif len(raw_timestamp) != 4:
84046
6b189f29b1e6 tweak exception message (again)
Brett Cannon <brett@python.org>
parents: 84023
diff changeset
676 message = 'reached EOF while reading timestamp in {!r}'.format(name)
81432
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
677 _verbose_message(message)
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
678 raise EOFError(message)
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
679 elif len(raw_size) != 4:
84046
6b189f29b1e6 tweak exception message (again)
Brett Cannon <brett@python.org>
parents: 84023
diff changeset
680 message = 'reached EOF while reading size of source in {!r}'.format(name)
81432
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
681 _verbose_message(message)
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
682 raise EOFError(message)
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
683 if source_stats is not None:
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
684 try:
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
685 source_mtime = int(source_stats['mtime'])
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
686 except KeyError:
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
687 pass
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
688 else:
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
689 if _r_long(raw_timestamp) != source_mtime:
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
690 message = 'bytecode is stale for {!r}'.format(name)
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
691 _verbose_message(message)
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
692 raise ImportError(message, **exc_details)
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
693 try:
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
694 source_size = source_stats['size'] & 0xFFFFFFFF
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
695 except KeyError:
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
696 pass
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
697 else:
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
698 if _r_long(raw_size) != source_size:
84582
33d379c2bb79 Issue #18415: Normalize what type of quotes are used with string
Brett Cannon <brett@python.org>
parents: 84468
diff changeset
699 raise ImportError('bytecode is stale for {!r}'.format(name),
81432
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
700 **exc_details)
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
701 return data[12:]
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
702
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
703
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
704 def _compile_bytecode(data, name=None, bytecode_path=None, source_path=None):
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
705 """Compile bytecode as returned by _validate_bytecode_header()."""
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
706 code = marshal.loads(data)
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
707 if isinstance(code, _code_type):
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
708 _verbose_message('code object from {!r}', bytecode_path)
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
709 if source_path is not None:
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
710 _imp._fix_co_filename(code, source_path)
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
711 return code
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
712 else:
84582
33d379c2bb79 Issue #18415: Normalize what type of quotes are used with string
Brett Cannon <brett@python.org>
parents: 84468
diff changeset
713 raise ImportError('Non-code object in {!r}'.format(bytecode_path),
81432
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
714 name=name, path=bytecode_path)
52141
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now.
Brett Cannon <bcannon@gmail.com>
parents: 52140
diff changeset
715
81761
d4eb02b6aac9 Port py_compile over to importlib
Brett Cannon <brett@python.org>
parents: 81735
diff changeset
716 def _code_to_bytecode(code, mtime=0, source_size=0):
d4eb02b6aac9 Port py_compile over to importlib
Brett Cannon <brett@python.org>
parents: 81735
diff changeset
717 """Compile a code object into bytecode for writing out to a byte-compiled
d4eb02b6aac9 Port py_compile over to importlib
Brett Cannon <brett@python.org>
parents: 81735
diff changeset
718 file."""
84128
5619bc2d8207 Issue #18192: Introduce importlib.util.MAGIC_NUMBER and document the
Brett Cannon <brett@python.org>
parents: 84109
diff changeset
719 data = bytearray(MAGIC_NUMBER)
81761
d4eb02b6aac9 Port py_compile over to importlib
Brett Cannon <brett@python.org>
parents: 81735
diff changeset
720 data.extend(_w_long(mtime))
d4eb02b6aac9 Port py_compile over to importlib
Brett Cannon <brett@python.org>
parents: 81735
diff changeset
721 data.extend(_w_long(source_size))
d4eb02b6aac9 Port py_compile over to importlib
Brett Cannon <brett@python.org>
parents: 81735
diff changeset
722 data.extend(marshal.dumps(code))
d4eb02b6aac9 Port py_compile over to importlib
Brett Cannon <brett@python.org>
parents: 81735
diff changeset
723 return data
52141
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now.
Brett Cannon <bcannon@gmail.com>
parents: 52140
diff changeset
724
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now.
Brett Cannon <bcannon@gmail.com>
parents: 52140
diff changeset
725
84173
bdd60bedf933 Issue #18076: Introduce imoportlib.util.decode_source().
Brett Cannon <brett@python.org>
parents: 84172
diff changeset
726 def decode_source(source_bytes):
bdd60bedf933 Issue #18076: Introduce imoportlib.util.decode_source().
Brett Cannon <brett@python.org>
parents: 84172
diff changeset
727 """Decode bytes representing source code and return the string.
bdd60bedf933 Issue #18076: Introduce imoportlib.util.decode_source().
Brett Cannon <brett@python.org>
parents: 84172
diff changeset
728
bdd60bedf933 Issue #18076: Introduce imoportlib.util.decode_source().
Brett Cannon <brett@python.org>
parents: 84172
diff changeset
729 Universal newline support is used in the decoding.
bdd60bedf933 Issue #18076: Introduce imoportlib.util.decode_source().
Brett Cannon <brett@python.org>
parents: 84172
diff changeset
730 """
bdd60bedf933 Issue #18076: Introduce imoportlib.util.decode_source().
Brett Cannon <brett@python.org>
parents: 84172
diff changeset
731 import tokenize # To avoid bootstrap issues.
bdd60bedf933 Issue #18076: Introduce imoportlib.util.decode_source().
Brett Cannon <brett@python.org>
parents: 84172
diff changeset
732 source_bytes_readline = _io.BytesIO(source_bytes).readline
bdd60bedf933 Issue #18076: Introduce imoportlib.util.decode_source().
Brett Cannon <brett@python.org>
parents: 84172
diff changeset
733 encoding = tokenize.detect_encoding(source_bytes_readline)
bdd60bedf933 Issue #18076: Introduce imoportlib.util.decode_source().
Brett Cannon <brett@python.org>
parents: 84172
diff changeset
734 newline_decoder = _io.IncrementalNewlineDecoder(None, True)
bdd60bedf933 Issue #18076: Introduce imoportlib.util.decode_source().
Brett Cannon <brett@python.org>
parents: 84172
diff changeset
735 return newline_decoder.decode(source_bytes.decode(encoding[0]))
52141
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now.
Brett Cannon <bcannon@gmail.com>
parents: 52140
diff changeset
736
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now.
Brett Cannon <bcannon@gmail.com>
parents: 52140
diff changeset
737
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now.
Brett Cannon <bcannon@gmail.com>
parents: 52140
diff changeset
738 # Loaders #####################################################################
52140
1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap.
Brett Cannon <bcannon@gmail.com>
parents: 52139
diff changeset
739
50983
0940a4f2506e Add importlib.machinery with its first tenants, BuitinImporter and
Brett Cannon <bcannon@gmail.com>
parents: 50893
diff changeset
740 class BuiltinImporter:
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
741
52165
62f9dad628de A few more docstring/API cleanups for importlib.
Brett Cannon <bcannon@gmail.com>
parents: 52164
diff changeset
742 """Meta path import for built-in modules.
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
743
52165
62f9dad628de A few more docstring/API cleanups for importlib.
Brett Cannon <bcannon@gmail.com>
parents: 52164
diff changeset
744 All methods are either class or static methods to avoid the need to
62f9dad628de A few more docstring/API cleanups for importlib.
Brett Cannon <bcannon@gmail.com>
parents: 52164
diff changeset
745 instantiate the class.
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
746
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
747 """
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
748
50983
0940a4f2506e Add importlib.machinery with its first tenants, BuitinImporter and
Brett Cannon <bcannon@gmail.com>
parents: 50893
diff changeset
749 @classmethod
77126
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
750 def module_repr(cls, module):
84582
33d379c2bb79 Issue #18415: Normalize what type of quotes are used with string
Brett Cannon <brett@python.org>
parents: 84468
diff changeset
751 return '<module {!r} (built-in)>'.format(module.__name__)
77126
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
752
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
753 @classmethod
50983
0940a4f2506e Add importlib.machinery with its first tenants, BuitinImporter and
Brett Cannon <bcannon@gmail.com>
parents: 50893
diff changeset
754 def find_module(cls, fullname, path=None):
52165
62f9dad628de A few more docstring/API cleanups for importlib.
Brett Cannon <bcannon@gmail.com>
parents: 52164
diff changeset
755 """Find the built-in module.
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
756
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
757 If 'path' is ever specified then the search is considered a failure.
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
758
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
759 """
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
760 if path is not None:
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
761 return None
76331
d777f854a66e Issue #13959: Rename imp to _imp and add Lib/imp.py and begin
Brett Cannon <brett@python.org>
parents: 76330
diff changeset
762 return cls if _imp.is_builtin(fullname) else None
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
763
51026
c26f1af26bfc Document both importlib.machinery.BuiltinImporter and FrozenImporter.
Brett Cannon <bcannon@gmail.com>
parents: 50983
diff changeset
764 @classmethod
52002
76a579b6bade Rename importlib.util.set___package__ to set_package.
Brett Cannon <bcannon@gmail.com>
parents: 51995
diff changeset
765 @set_package
52115
afd7118c8ccf Implement importlib.util.set_loader: a decorator to automatically set
Brett Cannon <bcannon@gmail.com>
parents: 52112
diff changeset
766 @set_loader
52166
69cd615b76fb Implement InspectLoader for BuiltinImporter.
Brett Cannon <bcannon@gmail.com>
parents: 52165
diff changeset
767 @_requires_builtin
51026
c26f1af26bfc Document both importlib.machinery.BuiltinImporter and FrozenImporter.
Brett Cannon <bcannon@gmail.com>
parents: 50983
diff changeset
768 def load_module(cls, fullname):
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
769 """Load a built-in module."""
84171
205aa49e5cd5 Issue #18115: Abstract out managing the cleanup of modules to use in
Brett Cannon <brett@python.org>
parents: 84169
diff changeset
770 with _ManageReload(fullname):
78344
62033490ca0f Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise
Nick Coghlan <ncoghlan@gmail.com>
parents: 78314
diff changeset
771 return _call_with_frames_removed(_imp.init_builtin, fullname)
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
772
52166
69cd615b76fb Implement InspectLoader for BuiltinImporter.
Brett Cannon <bcannon@gmail.com>
parents: 52165
diff changeset
773 @classmethod
69cd615b76fb Implement InspectLoader for BuiltinImporter.
Brett Cannon <bcannon@gmail.com>
parents: 52165
diff changeset
774 @_requires_builtin
69cd615b76fb Implement InspectLoader for BuiltinImporter.
Brett Cannon <bcannon@gmail.com>
parents: 52165
diff changeset
775 def get_code(cls, fullname):
69cd615b76fb Implement InspectLoader for BuiltinImporter.
Brett Cannon <bcannon@gmail.com>
parents: 52165
diff changeset
776 """Return None as built-in modules do not have code objects."""
69cd615b76fb Implement InspectLoader for BuiltinImporter.
Brett Cannon <bcannon@gmail.com>
parents: 52165
diff changeset
777 return None
69cd615b76fb Implement InspectLoader for BuiltinImporter.
Brett Cannon <bcannon@gmail.com>
parents: 52165
diff changeset
778
69cd615b76fb Implement InspectLoader for BuiltinImporter.
Brett Cannon <bcannon@gmail.com>
parents: 52165
diff changeset
779 @classmethod
69cd615b76fb Implement InspectLoader for BuiltinImporter.
Brett Cannon <bcannon@gmail.com>
parents: 52165
diff changeset
780 @_requires_builtin
69cd615b76fb Implement InspectLoader for BuiltinImporter.
Brett Cannon <bcannon@gmail.com>
parents: 52165
diff changeset
781 def get_source(cls, fullname):
69cd615b76fb Implement InspectLoader for BuiltinImporter.
Brett Cannon <bcannon@gmail.com>
parents: 52165
diff changeset
782 """Return None as built-in modules do not have source code."""
69cd615b76fb Implement InspectLoader for BuiltinImporter.
Brett Cannon <bcannon@gmail.com>
parents: 52165
diff changeset
783 return None
69cd615b76fb Implement InspectLoader for BuiltinImporter.
Brett Cannon <bcannon@gmail.com>
parents: 52165
diff changeset
784
69cd615b76fb Implement InspectLoader for BuiltinImporter.
Brett Cannon <bcannon@gmail.com>
parents: 52165
diff changeset
785 @classmethod
69cd615b76fb Implement InspectLoader for BuiltinImporter.
Brett Cannon <bcannon@gmail.com>
parents: 52165
diff changeset
786 @_requires_builtin
69cd615b76fb Implement InspectLoader for BuiltinImporter.
Brett Cannon <bcannon@gmail.com>
parents: 52165
diff changeset
787 def is_package(cls, fullname):
77964
37850f083af5 Minor refactoring in importlib._bootstrap, and fix the '_wrap' docstring.
Florent Xicluna <florent.xicluna@gmail.com>
parents: 77917
diff changeset
788 """Return False as built-in modules are never packages."""
52166
69cd615b76fb Implement InspectLoader for BuiltinImporter.
Brett Cannon <bcannon@gmail.com>
parents: 52165
diff changeset
789 return False
69cd615b76fb Implement InspectLoader for BuiltinImporter.
Brett Cannon <bcannon@gmail.com>
parents: 52165
diff changeset
790
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
791
50983
0940a4f2506e Add importlib.machinery with its first tenants, BuitinImporter and
Brett Cannon <bcannon@gmail.com>
parents: 50893
diff changeset
792 class FrozenImporter:
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
793
52165
62f9dad628de A few more docstring/API cleanups for importlib.
Brett Cannon <bcannon@gmail.com>
parents: 52164
diff changeset
794 """Meta path import for frozen modules.
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
795
52165
62f9dad628de A few more docstring/API cleanups for importlib.
Brett Cannon <bcannon@gmail.com>
parents: 52164
diff changeset
796 All methods are either class or static methods to avoid the need to
62f9dad628de A few more docstring/API cleanups for importlib.
Brett Cannon <bcannon@gmail.com>
parents: 52164
diff changeset
797 instantiate the class.
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
798
50983
0940a4f2506e Add importlib.machinery with its first tenants, BuitinImporter and
Brett Cannon <bcannon@gmail.com>
parents: 50893
diff changeset
799 """
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
800
50983
0940a4f2506e Add importlib.machinery with its first tenants, BuitinImporter and
Brett Cannon <bcannon@gmail.com>
parents: 50893
diff changeset
801 @classmethod
77126
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
802 def module_repr(cls, m):
84582
33d379c2bb79 Issue #18415: Normalize what type of quotes are used with string
Brett Cannon <brett@python.org>
parents: 84468
diff changeset
803 return '<module {!r} (frozen)>'.format(m.__name__)
77126
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
804
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
805 @classmethod
50983
0940a4f2506e Add importlib.machinery with its first tenants, BuitinImporter and
Brett Cannon <bcannon@gmail.com>
parents: 50893
diff changeset
806 def find_module(cls, fullname, path=None):
0940a4f2506e Add importlib.machinery with its first tenants, BuitinImporter and
Brett Cannon <bcannon@gmail.com>
parents: 50893
diff changeset
807 """Find a frozen module."""
76331
d777f854a66e Issue #13959: Rename imp to _imp and add Lib/imp.py and begin
Brett Cannon <brett@python.org>
parents: 76330
diff changeset
808 return cls if _imp.is_frozen(fullname) else None
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
809
50983
0940a4f2506e Add importlib.machinery with its first tenants, BuitinImporter and
Brett Cannon <bcannon@gmail.com>
parents: 50893
diff changeset
810 @classmethod
52002
76a579b6bade Rename importlib.util.set___package__ to set_package.
Brett Cannon <bcannon@gmail.com>
parents: 51995
diff changeset
811 @set_package
52115
afd7118c8ccf Implement importlib.util.set_loader: a decorator to automatically set
Brett Cannon <bcannon@gmail.com>
parents: 52112
diff changeset
812 @set_loader
52168
70f5f9113ae1 Implement InspectLoader for FrozenImporter.
Brett Cannon <bcannon@gmail.com>
parents: 52166
diff changeset
813 @_requires_frozen
50983
0940a4f2506e Add importlib.machinery with its first tenants, BuitinImporter and
Brett Cannon <bcannon@gmail.com>
parents: 50893
diff changeset
814 def load_module(cls, fullname):
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
815 """Load a frozen module."""
84171
205aa49e5cd5 Issue #18115: Abstract out managing the cleanup of modules to use in
Brett Cannon <brett@python.org>
parents: 84169
diff changeset
816 with _ManageReload(fullname):
78344
62033490ca0f Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise
Nick Coghlan <ncoghlan@gmail.com>
parents: 78314
diff changeset
817 m = _call_with_frames_removed(_imp.init_frozen, fullname)
77126
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
818 # Let our own module_repr() method produce a suitable repr.
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
819 del m.__file__
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
820 return m
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
821
52168
70f5f9113ae1 Implement InspectLoader for FrozenImporter.
Brett Cannon <bcannon@gmail.com>
parents: 52166
diff changeset
822 @classmethod
70f5f9113ae1 Implement InspectLoader for FrozenImporter.
Brett Cannon <bcannon@gmail.com>
parents: 52166
diff changeset
823 @_requires_frozen
70f5f9113ae1 Implement InspectLoader for FrozenImporter.
Brett Cannon <bcannon@gmail.com>
parents: 52166
diff changeset
824 def get_code(cls, fullname):
70f5f9113ae1 Implement InspectLoader for FrozenImporter.
Brett Cannon <bcannon@gmail.com>
parents: 52166
diff changeset
825 """Return the code object for the frozen module."""
76331
d777f854a66e Issue #13959: Rename imp to _imp and add Lib/imp.py and begin
Brett Cannon <brett@python.org>
parents: 76330
diff changeset
826 return _imp.get_frozen_object(fullname)
52168
70f5f9113ae1 Implement InspectLoader for FrozenImporter.
Brett Cannon <bcannon@gmail.com>
parents: 52166
diff changeset
827
70f5f9113ae1 Implement InspectLoader for FrozenImporter.
Brett Cannon <bcannon@gmail.com>
parents: 52166
diff changeset
828 @classmethod
70f5f9113ae1 Implement InspectLoader for FrozenImporter.
Brett Cannon <bcannon@gmail.com>
parents: 52166
diff changeset
829 @_requires_frozen
70f5f9113ae1 Implement InspectLoader for FrozenImporter.
Brett Cannon <bcannon@gmail.com>
parents: 52166
diff changeset
830 def get_source(cls, fullname):
70f5f9113ae1 Implement InspectLoader for FrozenImporter.
Brett Cannon <bcannon@gmail.com>
parents: 52166
diff changeset
831 """Return None as frozen modules do not have source code."""
70f5f9113ae1 Implement InspectLoader for FrozenImporter.
Brett Cannon <bcannon@gmail.com>
parents: 52166
diff changeset
832 return None
70f5f9113ae1 Implement InspectLoader for FrozenImporter.
Brett Cannon <bcannon@gmail.com>
parents: 52166
diff changeset
833
70f5f9113ae1 Implement InspectLoader for FrozenImporter.
Brett Cannon <bcannon@gmail.com>
parents: 52166
diff changeset
834 @classmethod
70f5f9113ae1 Implement InspectLoader for FrozenImporter.
Brett Cannon <bcannon@gmail.com>
parents: 52166
diff changeset
835 @_requires_frozen
70f5f9113ae1 Implement InspectLoader for FrozenImporter.
Brett Cannon <bcannon@gmail.com>
parents: 52166
diff changeset
836 def is_package(cls, fullname):
78491
f7b59e890e30 fix docstring wording
Philip Jenvey <pjenvey@underboss.org>
parents: 78489
diff changeset
837 """Return True if the frozen module is a package."""
76331
d777f854a66e Issue #13959: Rename imp to _imp and add Lib/imp.py and begin
Brett Cannon <brett@python.org>
parents: 76330
diff changeset
838 return _imp.is_frozen_package(fullname)
52168
70f5f9113ae1 Implement InspectLoader for FrozenImporter.
Brett Cannon <bcannon@gmail.com>
parents: 52166
diff changeset
839
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
840
78380
a1ac1e13c5a0 Close #15519: Properly expose WindowsRegistryFinder in importlib and bring the name into line with normal import terminology. Original patch by Eric Snow
Nick Coghlan <ncoghlan@gmail.com>
parents: 78344
diff changeset
841 class WindowsRegistryFinder:
78308
bd58c421057c Issue #14578: Support modules registered in the Windows registry again.
Martin v. Löwis <martin@v.loewis.de>
parents: 78112
diff changeset
842
78380
a1ac1e13c5a0 Close #15519: Properly expose WindowsRegistryFinder in importlib and bring the name into line with normal import terminology. Original patch by Eric Snow
Nick Coghlan <ncoghlan@gmail.com>
parents: 78344
diff changeset
843 """Meta path finder for modules declared in the Windows registry.
78308
bd58c421057c Issue #14578: Support modules registered in the Windows registry again.
Martin v. Löwis <martin@v.loewis.de>
parents: 78112
diff changeset
844 """
bd58c421057c Issue #14578: Support modules registered in the Windows registry again.
Martin v. Löwis <martin@v.loewis.de>
parents: 78112
diff changeset
845
bd58c421057c Issue #14578: Support modules registered in the Windows registry again.
Martin v. Löwis <martin@v.loewis.de>
parents: 78112
diff changeset
846 REGISTRY_KEY = (
84582
33d379c2bb79 Issue #18415: Normalize what type of quotes are used with string
Brett Cannon <brett@python.org>
parents: 84468
diff changeset
847 'Software\\Python\\PythonCore\\{sys_version}'
33d379c2bb79 Issue #18415: Normalize what type of quotes are used with string
Brett Cannon <brett@python.org>
parents: 84468
diff changeset
848 '\\Modules\\{fullname}')
78308
bd58c421057c Issue #14578: Support modules registered in the Windows registry again.
Martin v. Löwis <martin@v.loewis.de>
parents: 78112
diff changeset
849 REGISTRY_KEY_DEBUG = (
84582
33d379c2bb79 Issue #18415: Normalize what type of quotes are used with string
Brett Cannon <brett@python.org>
parents: 84468
diff changeset
850 'Software\\Python\\PythonCore\\{sys_version}'
33d379c2bb79 Issue #18415: Normalize what type of quotes are used with string
Brett Cannon <brett@python.org>
parents: 84468
diff changeset
851 '\\Modules\\{fullname}\\Debug')
78308
bd58c421057c Issue #14578: Support modules registered in the Windows registry again.
Martin v. Löwis <martin@v.loewis.de>
parents: 78112
diff changeset
852 DEBUG_BUILD = False # Changed in _setup()
bd58c421057c Issue #14578: Support modules registered in the Windows registry again.
Martin v. Löwis <martin@v.loewis.de>
parents: 78112
diff changeset
853
bd58c421057c Issue #14578: Support modules registered in the Windows registry again.
Martin v. Löwis <martin@v.loewis.de>
parents: 78112
diff changeset
854 @classmethod
bd58c421057c Issue #14578: Support modules registered in the Windows registry again.
Martin v. Löwis <martin@v.loewis.de>
parents: 78112
diff changeset
855 def _open_registry(cls, key):
bd58c421057c Issue #14578: Support modules registered in the Windows registry again.
Martin v. Löwis <martin@v.loewis.de>
parents: 78112
diff changeset
856 try:
bd58c421057c Issue #14578: Support modules registered in the Windows registry again.
Martin v. Löwis <martin@v.loewis.de>
parents: 78112
diff changeset
857 return _winreg.OpenKey(_winreg.HKEY_CURRENT_USER, key)
80944
bb94f6222fef Issue #16719: Get rid of WindowsError. Use OSError instead
Andrew Svetlov <andrew.svetlov@gmail.com>
parents: 80930
diff changeset
858 except OSError:
78308
bd58c421057c Issue #14578: Support modules registered in the Windows registry again.
Martin v. Löwis <martin@v.loewis.de>
parents: 78112
diff changeset
859 return _winreg.OpenKey(_winreg.HKEY_LOCAL_MACHINE, key)
bd58c421057c Issue #14578: Support modules registered in the Windows registry again.
Martin v. Löwis <martin@v.loewis.de>
parents: 78112
diff changeset
860
bd58c421057c Issue #14578: Support modules registered in the Windows registry again.
Martin v. Löwis <martin@v.loewis.de>
parents: 78112
diff changeset
861 @classmethod
bd58c421057c Issue #14578: Support modules registered in the Windows registry again.
Martin v. Löwis <martin@v.loewis.de>
parents: 78112
diff changeset
862 def _search_registry(cls, fullname):
bd58c421057c Issue #14578: Support modules registered in the Windows registry again.
Martin v. Löwis <martin@v.loewis.de>
parents: 78112
diff changeset
863 if cls.DEBUG_BUILD:
bd58c421057c Issue #14578: Support modules registered in the Windows registry again.
Martin v. Löwis <martin@v.loewis.de>
parents: 78112
diff changeset
864 registry_key = cls.REGISTRY_KEY_DEBUG
bd58c421057c Issue #14578: Support modules registered in the Windows registry again.
Martin v. Löwis <martin@v.loewis.de>
parents: 78112
diff changeset
865 else:
bd58c421057c Issue #14578: Support modules registered in the Windows registry again.
Martin v. Löwis <martin@v.loewis.de>
parents: 78112
diff changeset
866 registry_key = cls.REGISTRY_KEY
bd58c421057c Issue #14578: Support modules registered in the Windows registry again.
Martin v. Löwis <martin@v.loewis.de>
parents: 78112
diff changeset
867 key = registry_key.format(fullname=fullname,
bd58c421057c Issue #14578: Support modules registered in the Windows registry again.
Martin v. Löwis <martin@v.loewis.de>
parents: 78112
diff changeset
868 sys_version=sys.version[:3])
bd58c421057c Issue #14578: Support modules registered in the Windows registry again.
Martin v. Löwis <martin@v.loewis.de>
parents: 78112
diff changeset
869 try:
bd58c421057c Issue #14578: Support modules registered in the Windows registry again.
Martin v. Löwis <martin@v.loewis.de>
parents: 78112
diff changeset
870 with cls._open_registry(key) as hkey:
84582
33d379c2bb79 Issue #18415: Normalize what type of quotes are used with string
Brett Cannon <brett@python.org>
parents: 84468
diff changeset
871 filepath = _winreg.QueryValue(hkey, '')
80944
bb94f6222fef Issue #16719: Get rid of WindowsError. Use OSError instead
Andrew Svetlov <andrew.svetlov@gmail.com>
parents: 80930
diff changeset
872 except OSError:
78308
bd58c421057c Issue #14578: Support modules registered in the Windows registry again.
Martin v. Löwis <martin@v.loewis.de>
parents: 78112
diff changeset
873 return None
bd58c421057c Issue #14578: Support modules registered in the Windows registry again.
Martin v. Löwis <martin@v.loewis.de>
parents: 78112
diff changeset
874 return filepath
bd58c421057c Issue #14578: Support modules registered in the Windows registry again.
Martin v. Löwis <martin@v.loewis.de>
parents: 78112
diff changeset
875
bd58c421057c Issue #14578: Support modules registered in the Windows registry again.
Martin v. Löwis <martin@v.loewis.de>
parents: 78112
diff changeset
876 @classmethod
bd58c421057c Issue #14578: Support modules registered in the Windows registry again.
Martin v. Löwis <martin@v.loewis.de>
parents: 78112
diff changeset
877 def find_module(cls, fullname, path=None):
bd58c421057c Issue #14578: Support modules registered in the Windows registry again.
Martin v. Löwis <martin@v.loewis.de>
parents: 78112
diff changeset
878 """Find module named in the registry."""
bd58c421057c Issue #14578: Support modules registered in the Windows registry again.
Martin v. Löwis <martin@v.loewis.de>
parents: 78112
diff changeset
879 filepath = cls._search_registry(fullname)
bd58c421057c Issue #14578: Support modules registered in the Windows registry again.
Martin v. Löwis <martin@v.loewis.de>
parents: 78112
diff changeset
880 if filepath is None:
bd58c421057c Issue #14578: Support modules registered in the Windows registry again.
Martin v. Löwis <martin@v.loewis.de>
parents: 78112
diff changeset
881 return None
bd58c421057c Issue #14578: Support modules registered in the Windows registry again.
Martin v. Löwis <martin@v.loewis.de>
parents: 78112
diff changeset
882 try:
bd58c421057c Issue #14578: Support modules registered in the Windows registry again.
Martin v. Löwis <martin@v.loewis.de>
parents: 78112
diff changeset
883 _os.stat(filepath)
bd58c421057c Issue #14578: Support modules registered in the Windows registry again.
Martin v. Löwis <martin@v.loewis.de>
parents: 78112
diff changeset
884 except OSError:
bd58c421057c Issue #14578: Support modules registered in the Windows registry again.
Martin v. Löwis <martin@v.loewis.de>
parents: 78112
diff changeset
885 return None
85945
32b18998a560 [issue19151] Fix docstring and use of _get_suppported_file_loaders() to reflect 2-tuples.
Eric Snow <ericsnowcurrently@gmail.com>
parents: 85518
diff changeset
886 for loader, suffixes in _get_supported_file_loaders():
78308
bd58c421057c Issue #14578: Support modules registered in the Windows registry again.
Martin v. Löwis <martin@v.loewis.de>
parents: 78112
diff changeset
887 if filepath.endswith(tuple(suffixes)):
bd58c421057c Issue #14578: Support modules registered in the Windows registry again.
Martin v. Löwis <martin@v.loewis.de>
parents: 78112
diff changeset
888 return loader(fullname, filepath)
52168
70f5f9113ae1 Implement InspectLoader for FrozenImporter.
Brett Cannon <bcannon@gmail.com>
parents: 52166
diff changeset
889
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
890
62756
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib.
Brett Cannon <bcannon@gmail.com>
parents: 62587
diff changeset
891 class _LoaderBasics:
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib.
Brett Cannon <bcannon@gmail.com>
parents: 62587
diff changeset
892
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib.
Brett Cannon <bcannon@gmail.com>
parents: 62587
diff changeset
893 """Base class of common code needed by both SourceLoader and
76478
1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and
Brett Cannon <brett@python.org>
parents: 76472
diff changeset
894 SourcelessFileLoader."""
51771
ee7029710c05 Refactor source and bytecode file loaders in importlib so that there
Brett Cannon <bcannon@gmail.com>
parents: 51770
diff changeset
895
62756
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib.
Brett Cannon <bcannon@gmail.com>
parents: 62587
diff changeset
896 def is_package(self, fullname):
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib.
Brett Cannon <bcannon@gmail.com>
parents: 62587
diff changeset
897 """Concrete implementation of InspectLoader.is_package by checking if
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib.
Brett Cannon <bcannon@gmail.com>
parents: 62587
diff changeset
898 the path returned by get_filename has a filename of '__init__.py'."""
76449
163360370ad2 Make path manipulation more robust for platforms with alternative path
Brett Cannon <brett@python.org>
parents: 76445
diff changeset
899 filename = _path_split(self.get_filename(fullname))[1]
77457
240b7467e65c Issue #14938: importlib.abc.SourceLoader.is_package() now takes the
Brett Cannon <brett@python.org>
parents: 77180
diff changeset
900 filename_base = filename.rsplit('.', 1)[0]
240b7467e65c Issue #14938: importlib.abc.SourceLoader.is_package() now takes the
Brett Cannon <brett@python.org>
parents: 77180
diff changeset
901 tail_name = fullname.rpartition('.')[2]
240b7467e65c Issue #14938: importlib.abc.SourceLoader.is_package() now takes the
Brett Cannon <brett@python.org>
parents: 77180
diff changeset
902 return filename_base == '__init__' and tail_name != '__init__'
52165
62f9dad628de A few more docstring/API cleanups for importlib.
Brett Cannon <bcannon@gmail.com>
parents: 52164
diff changeset
903
84001
e873f2e67353 Issues #18088, 18089: Introduce
Brett Cannon <brett@python.org>
parents: 83999
diff changeset
904 def init_module_attrs(self, module):
e873f2e67353 Issues #18088, 18089: Introduce
Brett Cannon <brett@python.org>
parents: 83999
diff changeset
905 """Set various attributes on the module.
51771
ee7029710c05 Refactor source and bytecode file loaders in importlib so that there
Brett Cannon <bcannon@gmail.com>
parents: 51770
diff changeset
906
84001
e873f2e67353 Issues #18088, 18089: Introduce
Brett Cannon <brett@python.org>
parents: 83999
diff changeset
907 ExecutionLoader.init_module_attrs() is used to set __loader__,
e873f2e67353 Issues #18088, 18089: Introduce
Brett Cannon <brett@python.org>
parents: 83999
diff changeset
908 __package__, __file__, and optionally __path__. The __cached__ attribute
e873f2e67353 Issues #18088, 18089: Introduce
Brett Cannon <brett@python.org>
parents: 83999
diff changeset
909 is set using imp.cache_from_source() and __file__.
62756
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib.
Brett Cannon <bcannon@gmail.com>
parents: 62587
diff changeset
910 """
84001
e873f2e67353 Issues #18088, 18089: Introduce
Brett Cannon <brett@python.org>
parents: 83999
diff changeset
911 module.__loader__ = self # Loader
e873f2e67353 Issues #18088, 18089: Introduce
Brett Cannon <brett@python.org>
parents: 83999
diff changeset
912 _init_package_attrs(self, module) # InspectLoader
e873f2e67353 Issues #18088, 18089: Introduce
Brett Cannon <brett@python.org>
parents: 83999
diff changeset
913 _init_file_attrs(self, module) # ExecutionLoader
e873f2e67353 Issues #18088, 18089: Introduce
Brett Cannon <brett@python.org>
parents: 83999
diff changeset
914 if hasattr(module, '__file__'): # SourceLoader
78026
e86330669bb5 Issue #15056: imp.cache_from_source() and source_from_cache() raise
Brett Cannon <brett@python.org>
parents: 78010
diff changeset
915 try:
e86330669bb5 Issue #15056: imp.cache_from_source() and source_from_cache() raise
Brett Cannon <brett@python.org>
parents: 78010
diff changeset
916 module.__cached__ = cache_from_source(module.__file__)
e86330669bb5 Issue #15056: imp.cache_from_source() and source_from_cache() raise
Brett Cannon <brett@python.org>
parents: 78010
diff changeset
917 except NotImplementedError:
84001
e873f2e67353 Issues #18088, 18089: Introduce
Brett Cannon <brett@python.org>
parents: 83999
diff changeset
918 pass
e873f2e67353 Issues #18088, 18089: Introduce
Brett Cannon <brett@python.org>
parents: 83999
diff changeset
919
e873f2e67353 Issues #18088, 18089: Introduce
Brett Cannon <brett@python.org>
parents: 83999
diff changeset
920 def load_module(self, fullname):
e873f2e67353 Issues #18088, 18089: Introduce
Brett Cannon <brett@python.org>
parents: 83999
diff changeset
921 """Load the specified module into sys.modules and return it."""
e873f2e67353 Issues #18088, 18089: Introduce
Brett Cannon <brett@python.org>
parents: 83999
diff changeset
922 with module_to_load(fullname) as module:
e873f2e67353 Issues #18088, 18089: Introduce
Brett Cannon <brett@python.org>
parents: 83999
diff changeset
923 self.init_module_attrs(module)
e873f2e67353 Issues #18088, 18089: Introduce
Brett Cannon <brett@python.org>
parents: 83999
diff changeset
924 code = self.get_code(fullname)
e873f2e67353 Issues #18088, 18089: Introduce
Brett Cannon <brett@python.org>
parents: 83999
diff changeset
925 if code is None:
e873f2e67353 Issues #18088, 18089: Introduce
Brett Cannon <brett@python.org>
parents: 83999
diff changeset
926 raise ImportError('cannot load module {!r} when get_code() '
e873f2e67353 Issues #18088, 18089: Introduce
Brett Cannon <brett@python.org>
parents: 83999
diff changeset
927 'returns None'.format(fullname))
e873f2e67353 Issues #18088, 18089: Introduce
Brett Cannon <brett@python.org>
parents: 83999
diff changeset
928 _call_with_frames_removed(exec, code, module.__dict__)
e873f2e67353 Issues #18088, 18089: Introduce
Brett Cannon <brett@python.org>
parents: 83999
diff changeset
929 return module
51771
ee7029710c05 Refactor source and bytecode file loaders in importlib so that there
Brett Cannon <bcannon@gmail.com>
parents: 51770
diff changeset
930
62756
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib.
Brett Cannon <bcannon@gmail.com>
parents: 62587
diff changeset
931
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib.
Brett Cannon <bcannon@gmail.com>
parents: 62587
diff changeset
932 class SourceLoader(_LoaderBasics):
51771
ee7029710c05 Refactor source and bytecode file loaders in importlib so that there
Brett Cannon <bcannon@gmail.com>
parents: 51770
diff changeset
933
67573
e125d92f7438 Issue 10899: Remove function type annotations from the stdlib
Raymond Hettinger <python@rcn.com>
parents: 66226
diff changeset
934 def path_mtime(self, path):
67581
7d658b49b3e1 Issue 10899: Remove function type annotations from the stdlib
Raymond Hettinger <python@rcn.com>
parents: 67573
diff changeset
935 """Optional method that returns the modification time (an int) for the
7d658b49b3e1 Issue 10899: Remove function type annotations from the stdlib
Raymond Hettinger <python@rcn.com>
parents: 67573
diff changeset
936 specified path, where path is a str.
83225
8e733e30edf6 Issue #17093,17566,17567: Methods from classes in importlib.abc now raise/return
Brett Cannon <brett@python.org>
parents: 83056
diff changeset
937
8e733e30edf6 Issue #17093,17566,17567: Methods from classes in importlib.abc now raise/return
Brett Cannon <brett@python.org>
parents: 83056
diff changeset
938 Raises IOError when the path cannot be handled.
74358
87331661042b Issue #13645: pyc files now contain the size of the corresponding source
Antoine Pitrou <solipsis@pitrou.net>
parents: 73977
diff changeset
939 """
83225
8e733e30edf6 Issue #17093,17566,17567: Methods from classes in importlib.abc now raise/return
Brett Cannon <brett@python.org>
parents: 83056
diff changeset
940 raise IOError
74358
87331661042b Issue #13645: pyc files now contain the size of the corresponding source
Antoine Pitrou <solipsis@pitrou.net>
parents: 73977
diff changeset
941
87331661042b Issue #13645: pyc files now contain the size of the corresponding source
Antoine Pitrou <solipsis@pitrou.net>
parents: 73977
diff changeset
942 def path_stats(self, path):
87331661042b Issue #13645: pyc files now contain the size of the corresponding source
Antoine Pitrou <solipsis@pitrou.net>
parents: 73977
diff changeset
943 """Optional method returning a metadata dict for the specified path
87331661042b Issue #13645: pyc files now contain the size of the corresponding source
Antoine Pitrou <solipsis@pitrou.net>
parents: 73977
diff changeset
944 to by the path (str).
87331661042b Issue #13645: pyc files now contain the size of the corresponding source
Antoine Pitrou <solipsis@pitrou.net>
parents: 73977
diff changeset
945 Possible keys:
87331661042b Issue #13645: pyc files now contain the size of the corresponding source
Antoine Pitrou <solipsis@pitrou.net>
parents: 73977
diff changeset
946 - 'mtime' (mandatory) is the numeric timestamp of last source
87331661042b Issue #13645: pyc files now contain the size of the corresponding source
Antoine Pitrou <solipsis@pitrou.net>
parents: 73977
diff changeset
947 code modification;
87331661042b Issue #13645: pyc files now contain the size of the corresponding source
Antoine Pitrou <solipsis@pitrou.net>
parents: 73977
diff changeset
948 - 'size' (optional) is the size in bytes of the source code.
52112
03080fa08dfe Implement get_source for importlib.abc.PyLoader using source_path and get_data.
Brett Cannon <bcannon@gmail.com>
parents: 52083
diff changeset
949
62587
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap.
Brett Cannon <bcannon@gmail.com>
parents: 61692
diff changeset
950 Implementing this method allows the loader to read bytecode files.
83225
8e733e30edf6 Issue #17093,17566,17567: Methods from classes in importlib.abc now raise/return
Brett Cannon <brett@python.org>
parents: 83056
diff changeset
951 Raises IOError when the path cannot be handled.
52112
03080fa08dfe Implement get_source for importlib.abc.PyLoader using source_path and get_data.
Brett Cannon <bcannon@gmail.com>
parents: 52083
diff changeset
952 """
74358
87331661042b Issue #13645: pyc files now contain the size of the corresponding source
Antoine Pitrou <solipsis@pitrou.net>
parents: 73977
diff changeset
953 return {'mtime': self.path_mtime(path)}
62587
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap.
Brett Cannon <bcannon@gmail.com>
parents: 61692
diff changeset
954
78722
3a831a0a29c4 Close #2501: Permission bits are once again correctly copied from the source file to the cached bytecode file. Test by Eric Snow.
Nick Coghlan <ncoghlan@gmail.com>
parents: 78663
diff changeset
955 def _cache_bytecode(self, source_path, cache_path, data):
3a831a0a29c4 Close #2501: Permission bits are once again correctly copied from the source file to the cached bytecode file. Test by Eric Snow.
Nick Coghlan <ncoghlan@gmail.com>
parents: 78663
diff changeset
956 """Optional method which writes data (bytes) to a file path (a str).
3a831a0a29c4 Close #2501: Permission bits are once again correctly copied from the source file to the cached bytecode file. Test by Eric Snow.
Nick Coghlan <ncoghlan@gmail.com>
parents: 78663
diff changeset
957
3a831a0a29c4 Close #2501: Permission bits are once again correctly copied from the source file to the cached bytecode file. Test by Eric Snow.
Nick Coghlan <ncoghlan@gmail.com>
parents: 78663
diff changeset
958 Implementing this method allows for the writing of bytecode files.
3a831a0a29c4 Close #2501: Permission bits are once again correctly copied from the source file to the cached bytecode file. Test by Eric Snow.
Nick Coghlan <ncoghlan@gmail.com>
parents: 78663
diff changeset
959
3a831a0a29c4 Close #2501: Permission bits are once again correctly copied from the source file to the cached bytecode file. Test by Eric Snow.
Nick Coghlan <ncoghlan@gmail.com>
parents: 78663
diff changeset
960 The source path is needed in order to correctly transfer permissions
3a831a0a29c4 Close #2501: Permission bits are once again correctly copied from the source file to the cached bytecode file. Test by Eric Snow.
Nick Coghlan <ncoghlan@gmail.com>
parents: 78663
diff changeset
961 """
3a831a0a29c4 Close #2501: Permission bits are once again correctly copied from the source file to the cached bytecode file. Test by Eric Snow.
Nick Coghlan <ncoghlan@gmail.com>
parents: 78663
diff changeset
962 # For backwards compatibility, we delegate to set_data()
3a831a0a29c4 Close #2501: Permission bits are once again correctly copied from the source file to the cached bytecode file. Test by Eric Snow.
Nick Coghlan <ncoghlan@gmail.com>
parents: 78663
diff changeset
963 return self.set_data(cache_path, data)
62587
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap.
Brett Cannon <bcannon@gmail.com>
parents: 61692
diff changeset
964
67573
e125d92f7438 Issue 10899: Remove function type annotations from the stdlib
Raymond Hettinger <python@rcn.com>
parents: 66226
diff changeset
965 def set_data(self, path, data):
67581
7d658b49b3e1 Issue 10899: Remove function type annotations from the stdlib
Raymond Hettinger <python@rcn.com>
parents: 67573
diff changeset
966 """Optional method which writes data (bytes) to a file path (a str).
62587
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap.
Brett Cannon <bcannon@gmail.com>
parents: 61692
diff changeset
967
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap.
Brett Cannon <bcannon@gmail.com>
parents: 61692
diff changeset
968 Implementing this method allows for the writing of bytecode files.
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap.
Brett Cannon <bcannon@gmail.com>
parents: 61692
diff changeset
969 """
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap.
Brett Cannon <bcannon@gmail.com>
parents: 61692
diff changeset
970
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap.
Brett Cannon <bcannon@gmail.com>
parents: 61692
diff changeset
971
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap.
Brett Cannon <bcannon@gmail.com>
parents: 61692
diff changeset
972 def get_source(self, fullname):
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap.
Brett Cannon <bcannon@gmail.com>
parents: 61692
diff changeset
973 """Concrete implementation of InspectLoader.get_source."""
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap.
Brett Cannon <bcannon@gmail.com>
parents: 61692
diff changeset
974 path = self.get_filename(fullname)
52112
03080fa08dfe Implement get_source for importlib.abc.PyLoader using source_path and get_data.
Brett Cannon <bcannon@gmail.com>
parents: 52083
diff changeset
975 try:
03080fa08dfe Implement get_source for importlib.abc.PyLoader using source_path and get_data.
Brett Cannon <bcannon@gmail.com>
parents: 52083
diff changeset
976 source_bytes = self.get_data(path)
81039
7d69d04522e3 Replace IOError with OSError (#16715)
Andrew Svetlov <andrew.svetlov@gmail.com>
parents: 80944
diff changeset
977 except OSError as exc:
84582
33d379c2bb79 Issue #18415: Normalize what type of quotes are used with string
Brett Cannon <brett@python.org>
parents: 84468
diff changeset
978 raise ImportError('source not available through get_data()',
78112
7d202353a728 Issue #15343: A lot more than just unicode decoding can go wrong when retrieving a source file
Nick Coghlan <ncoghlan@gmail.com>
parents: 78078
diff changeset
979 name=fullname) from exc
84173
bdd60bedf933 Issue #18076: Introduce imoportlib.util.decode_source().
Brett Cannon <brett@python.org>
parents: 84172
diff changeset
980 return decode_source(source_bytes)
51771
ee7029710c05 Refactor source and bytecode file loaders in importlib so that there
Brett Cannon <bcannon@gmail.com>
parents: 51770
diff changeset
981
81761
d4eb02b6aac9 Port py_compile over to importlib
Brett Cannon <brett@python.org>
parents: 81735
diff changeset
982 def source_to_code(self, data, path, *, _optimize=-1):
80507
e30bcce5c634 Issue #15627: Add the compile_source() method to
Brett Cannon <brett@python.org>
parents: 80410
diff changeset
983 """Return the code object compiled from source.
e30bcce5c634 Issue #15627: Add the compile_source() method to
Brett Cannon <brett@python.org>
parents: 80410
diff changeset
984
e30bcce5c634 Issue #15627: Add the compile_source() method to
Brett Cannon <brett@python.org>
parents: 80410
diff changeset
985 The 'data' argument can be any object type that compile() supports.
e30bcce5c634 Issue #15627: Add the compile_source() method to
Brett Cannon <brett@python.org>
parents: 80410
diff changeset
986 """
e30bcce5c634 Issue #15627: Add the compile_source() method to
Brett Cannon <brett@python.org>
parents: 80410
diff changeset
987 return _call_with_frames_removed(compile, data, path, 'exec',
81761
d4eb02b6aac9 Port py_compile over to importlib
Brett Cannon <brett@python.org>
parents: 81735
diff changeset
988 dont_inherit=True, optimize=_optimize)
51771
ee7029710c05 Refactor source and bytecode file loaders in importlib so that there
Brett Cannon <bcannon@gmail.com>
parents: 51770
diff changeset
989
ee7029710c05 Refactor source and bytecode file loaders in importlib so that there
Brett Cannon <bcannon@gmail.com>
parents: 51770
diff changeset
990 def get_code(self, fullname):
62587
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap.
Brett Cannon <bcannon@gmail.com>
parents: 61692
diff changeset
991 """Concrete implementation of InspectLoader.get_code.
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap.
Brett Cannon <bcannon@gmail.com>
parents: 61692
diff changeset
992
74358
87331661042b Issue #13645: pyc files now contain the size of the corresponding source
Antoine Pitrou <solipsis@pitrou.net>
parents: 73977
diff changeset
993 Reading of bytecode requires path_stats to be implemented. To write
62587
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap.
Brett Cannon <bcannon@gmail.com>
parents: 61692
diff changeset
994 bytecode, set_data must also be implemented.
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap.
Brett Cannon <bcannon@gmail.com>
parents: 61692
diff changeset
995
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap.
Brett Cannon <bcannon@gmail.com>
parents: 61692
diff changeset
996 """
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap.
Brett Cannon <bcannon@gmail.com>
parents: 61692
diff changeset
997 source_path = self.get_filename(fullname)
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap.
Brett Cannon <bcannon@gmail.com>
parents: 61692
diff changeset
998 source_mtime = None
78026
e86330669bb5 Issue #15056: imp.cache_from_source() and source_from_cache() raise
Brett Cannon <brett@python.org>
parents: 78010
diff changeset
999 try:
e86330669bb5 Issue #15056: imp.cache_from_source() and source_from_cache() raise
Brett Cannon <brett@python.org>
parents: 78010
diff changeset
1000 bytecode_path = cache_from_source(source_path)
e86330669bb5 Issue #15056: imp.cache_from_source() and source_from_cache() raise
Brett Cannon <brett@python.org>
parents: 78010
diff changeset
1001 except NotImplementedError:
e86330669bb5 Issue #15056: imp.cache_from_source() and source_from_cache() raise
Brett Cannon <brett@python.org>
parents: 78010
diff changeset
1002 bytecode_path = None
e86330669bb5 Issue #15056: imp.cache_from_source() and source_from_cache() raise
Brett Cannon <brett@python.org>
parents: 78010
diff changeset
1003 else:
51771
ee7029710c05 Refactor source and bytecode file loaders in importlib so that there
Brett Cannon <bcannon@gmail.com>
parents: 51770
diff changeset
1004 try:
74358
87331661042b Issue #13645: pyc files now contain the size of the corresponding source
Antoine Pitrou <solipsis@pitrou.net>
parents: 73977
diff changeset
1005 st = self.path_stats(source_path)
83225
8e733e30edf6 Issue #17093,17566,17567: Methods from classes in importlib.abc now raise/return
Brett Cannon <brett@python.org>
parents: 83056
diff changeset
1006 except IOError:
62587
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap.
Brett Cannon <bcannon@gmail.com>
parents: 61692
diff changeset
1007 pass
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap.
Brett Cannon <bcannon@gmail.com>
parents: 61692
diff changeset
1008 else:
74358
87331661042b Issue #13645: pyc files now contain the size of the corresponding source
Antoine Pitrou <solipsis@pitrou.net>
parents: 73977
diff changeset
1009 source_mtime = int(st['mtime'])
62587
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap.
Brett Cannon <bcannon@gmail.com>
parents: 61692
diff changeset
1010 try:
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap.
Brett Cannon <bcannon@gmail.com>
parents: 61692
diff changeset
1011 data = self.get_data(bytecode_path)
81039
7d69d04522e3 Replace IOError with OSError (#16715)
Andrew Svetlov <andrew.svetlov@gmail.com>
parents: 80944
diff changeset
1012 except OSError:
51771
ee7029710c05 Refactor source and bytecode file loaders in importlib so that there
Brett Cannon <bcannon@gmail.com>
parents: 51770
diff changeset
1013 pass
ee7029710c05 Refactor source and bytecode file loaders in importlib so that there
Brett Cannon <bcannon@gmail.com>
parents: 51770
diff changeset
1014 else:
62756
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib.
Brett Cannon <bcannon@gmail.com>
parents: 62587
diff changeset
1015 try:
81432
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
1016 bytes_data = _validate_bytecode_header(data,
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
1017 source_stats=st, name=fullname,
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
1018 path=bytecode_path)
62756
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib.
Brett Cannon <bcannon@gmail.com>
parents: 62587
diff changeset
1019 except (ImportError, EOFError):
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib.
Brett Cannon <bcannon@gmail.com>
parents: 62587
diff changeset
1020 pass
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib.
Brett Cannon <bcannon@gmail.com>
parents: 62587
diff changeset
1021 else:
76753
59b8a7fcb047 Properly mark names in importlib._bootstrap as private.
Brett Cannon <brett@python.org>
parents: 76586
diff changeset
1022 _verbose_message('{} matches {}', bytecode_path,
76303
2dd046be2c88 Issue #2377: Make importlib the implementation of __import__().
Brett Cannon <brett@python.org>
parents: 76280
diff changeset
1023 source_path)
81432
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
1024 return _compile_bytecode(bytes_data, name=fullname,
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
1025 bytecode_path=bytecode_path,
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
1026 source_path=source_path)
62587
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap.
Brett Cannon <bcannon@gmail.com>
parents: 61692
diff changeset
1027 source_bytes = self.get_data(source_path)
80725
7caab48f171e Issue #15627: This is simply an update to the name of a new method recently added
Eric Snow <ericsnowcurrently@gmail.com>
parents: 80530
diff changeset
1028 code_object = self.source_to_code(source_bytes, source_path)
76753
59b8a7fcb047 Properly mark names in importlib._bootstrap as private.
Brett Cannon <brett@python.org>
parents: 76586
diff changeset
1029 _verbose_message('code object from {}', source_path)
62587
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap.
Brett Cannon <bcannon@gmail.com>
parents: 61692
diff changeset
1030 if (not sys.dont_write_bytecode and bytecode_path is not None and
81761
d4eb02b6aac9 Port py_compile over to importlib
Brett Cannon <brett@python.org>
parents: 81735
diff changeset
1031 source_mtime is not None):
d4eb02b6aac9 Port py_compile over to importlib
Brett Cannon <brett@python.org>
parents: 81735
diff changeset
1032 data = _code_to_bytecode(code_object, source_mtime,
d4eb02b6aac9 Port py_compile over to importlib
Brett Cannon <brett@python.org>
parents: 81735
diff changeset
1033 len(source_bytes))
62587
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap.
Brett Cannon <bcannon@gmail.com>
parents: 61692
diff changeset
1034 try:
78722
3a831a0a29c4 Close #2501: Permission bits are once again correctly copied from the source file to the cached bytecode file. Test by Eric Snow.
Nick Coghlan <ncoghlan@gmail.com>
parents: 78663
diff changeset
1035 self._cache_bytecode(source_path, bytecode_path, data)
76753
59b8a7fcb047 Properly mark names in importlib._bootstrap as private.
Brett Cannon <brett@python.org>
parents: 76586
diff changeset
1036 _verbose_message('wrote {!r}', bytecode_path)
62587
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap.
Brett Cannon <bcannon@gmail.com>
parents: 61692
diff changeset
1037 except NotImplementedError:
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap.
Brett Cannon <bcannon@gmail.com>
parents: 61692
diff changeset
1038 pass
51771
ee7029710c05 Refactor source and bytecode file loaders in importlib so that there
Brett Cannon <bcannon@gmail.com>
parents: 51770
diff changeset
1039 return code_object
ee7029710c05 Refactor source and bytecode file loaders in importlib so that there
Brett Cannon <bcannon@gmail.com>
parents: 51770
diff changeset
1040
ee7029710c05 Refactor source and bytecode file loaders in importlib so that there
Brett Cannon <bcannon@gmail.com>
parents: 51770
diff changeset
1041
76478
1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and
Brett Cannon <brett@python.org>
parents: 76472
diff changeset
1042 class FileLoader:
51771
ee7029710c05 Refactor source and bytecode file loaders in importlib so that there
Brett Cannon <bcannon@gmail.com>
parents: 51770
diff changeset
1043
62756
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib.
Brett Cannon <bcannon@gmail.com>
parents: 62587
diff changeset
1044 """Base file loader class which implements the loader protocol methods that
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib.
Brett Cannon <bcannon@gmail.com>
parents: 62587
diff changeset
1045 require file system usage."""
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
1046
62587
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap.
Brett Cannon <bcannon@gmail.com>
parents: 61692
diff changeset
1047 def __init__(self, fullname, path):
62756
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib.
Brett Cannon <bcannon@gmail.com>
parents: 62587
diff changeset
1048 """Cache the module name and the path to the file found by the
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib.
Brett Cannon <bcannon@gmail.com>
parents: 62587
diff changeset
1049 finder."""
76478
1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and
Brett Cannon <brett@python.org>
parents: 76472
diff changeset
1050 self.name = fullname
1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and
Brett Cannon <brett@python.org>
parents: 76472
diff changeset
1051 self.path = path
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
1052
52139
2e8e897a1244 Make utility code in importlib._bootstrap private.
Brett Cannon <bcannon@gmail.com>
parents: 52115
diff changeset
1053 @_check_name
76877
626d5c6fbd95 Issue #13959: Have
Brett Cannon <brett@python.org>
parents: 76874
diff changeset
1054 def load_module(self, fullname):
626d5c6fbd95 Issue #13959: Have
Brett Cannon <brett@python.org>
parents: 76874
diff changeset
1055 """Load a module from a file."""
77180
bcb3b81853cc Tweak importlib._bootstrap to avoid zero-argument super so I can work on issue #14857 without breaking imports
Nick Coghlan <ncoghlan@gmail.com>
parents: 77164
diff changeset
1056 # Issue #14857: Avoid the zero-argument form so the implementation
bcb3b81853cc Tweak importlib._bootstrap to avoid zero-argument super so I can work on issue #14857 without breaking imports
Nick Coghlan <ncoghlan@gmail.com>
parents: 77164
diff changeset
1057 # of that form can be updated without breaking the frozen module
bcb3b81853cc Tweak importlib._bootstrap to avoid zero-argument super so I can work on issue #14857 without breaking imports
Nick Coghlan <ncoghlan@gmail.com>
parents: 77164
diff changeset
1058 return super(FileLoader, self).load_module(fullname)
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
1059
52139
2e8e897a1244 Make utility code in importlib._bootstrap private.
Brett Cannon <bcannon@gmail.com>
parents: 52115
diff changeset
1060 @_check_name
62587
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap.
Brett Cannon <bcannon@gmail.com>
parents: 61692
diff changeset
1061 def get_filename(self, fullname):
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap.
Brett Cannon <bcannon@gmail.com>
parents: 61692
diff changeset
1062 """Return the path to the source file as found by the finder."""
76478
1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and
Brett Cannon <brett@python.org>
parents: 76472
diff changeset
1063 return self.path
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
1064
51771
ee7029710c05 Refactor source and bytecode file loaders in importlib so that there
Brett Cannon <bcannon@gmail.com>
parents: 51770
diff changeset
1065 def get_data(self, path):
ee7029710c05 Refactor source and bytecode file loaders in importlib so that there
Brett Cannon <bcannon@gmail.com>
parents: 51770
diff changeset
1066 """Return the data from path as raw bytes."""
64407
f8ba9558adbf Remove redundant context manager.
Florent Xicluna <florent.xicluna@gmail.com>
parents: 64291
diff changeset
1067 with _io.FileIO(path, 'r') as file:
62756
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib.
Brett Cannon <bcannon@gmail.com>
parents: 62587
diff changeset
1068 return file.read()
51771
ee7029710c05 Refactor source and bytecode file loaders in importlib so that there
Brett Cannon <bcannon@gmail.com>
parents: 51770
diff changeset
1069
ee7029710c05 Refactor source and bytecode file loaders in importlib so that there
Brett Cannon <bcannon@gmail.com>
parents: 51770
diff changeset
1070
76478
1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and
Brett Cannon <brett@python.org>
parents: 76472
diff changeset
1071 class SourceFileLoader(FileLoader, SourceLoader):
51771
ee7029710c05 Refactor source and bytecode file loaders in importlib so that there
Brett Cannon <bcannon@gmail.com>
parents: 51770
diff changeset
1072
62756
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib.
Brett Cannon <bcannon@gmail.com>
parents: 62587
diff changeset
1073 """Concrete implementation of SourceLoader using the file system."""
51771
ee7029710c05 Refactor source and bytecode file loaders in importlib so that there
Brett Cannon <bcannon@gmail.com>
parents: 51770
diff changeset
1074
74358
87331661042b Issue #13645: pyc files now contain the size of the corresponding source
Antoine Pitrou <solipsis@pitrou.net>
parents: 73977
diff changeset
1075 def path_stats(self, path):
78491
f7b59e890e30 fix docstring wording
Philip Jenvey <pjenvey@underboss.org>
parents: 78489
diff changeset
1076 """Return the metadata for the path."""
74358
87331661042b Issue #13645: pyc files now contain the size of the corresponding source
Antoine Pitrou <solipsis@pitrou.net>
parents: 73977
diff changeset
1077 st = _os.stat(path)
87331661042b Issue #13645: pyc files now contain the size of the corresponding source
Antoine Pitrou <solipsis@pitrou.net>
parents: 73977
diff changeset
1078 return {'mtime': st.st_mtime, 'size': st.st_size}
51771
ee7029710c05 Refactor source and bytecode file loaders in importlib so that there
Brett Cannon <bcannon@gmail.com>
parents: 51770
diff changeset
1079
78722
3a831a0a29c4 Close #2501: Permission bits are once again correctly copied from the source file to the cached bytecode file. Test by Eric Snow.
Nick Coghlan <ncoghlan@gmail.com>
parents: 78663
diff changeset
1080 def _cache_bytecode(self, source_path, bytecode_path, data):
3a831a0a29c4 Close #2501: Permission bits are once again correctly copied from the source file to the cached bytecode file. Test by Eric Snow.
Nick Coghlan <ncoghlan@gmail.com>
parents: 78663
diff changeset
1081 # Adapt between the two APIs
83376
04aaeae6ee7b Issue #17244: Don't mask exceptions raised during the creation of
Brett Cannon <brett@python.org>
parents: 83225
diff changeset
1082 mode = _calc_mode(source_path)
78738
0c661c5632e0 Issue #2051: Tweak last commit for this issue to pass in mode instead
Brett Cannon <brett@python.org>
parents: 78722
diff changeset
1083 return self.set_data(bytecode_path, data, _mode=mode)
78722
3a831a0a29c4 Close #2501: Permission bits are once again correctly copied from the source file to the cached bytecode file. Test by Eric Snow.
Nick Coghlan <ncoghlan@gmail.com>
parents: 78663
diff changeset
1084
78738
0c661c5632e0 Issue #2051: Tweak last commit for this issue to pass in mode instead
Brett Cannon <brett@python.org>
parents: 78722
diff changeset
1085 def set_data(self, path, data, *, _mode=0o666):
62587
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap.
Brett Cannon <bcannon@gmail.com>
parents: 61692
diff changeset
1086 """Write bytes data to a file."""
76449
163360370ad2 Make path manipulation more robust for platforms with alternative path
Brett Cannon <brett@python.org>
parents: 76445
diff changeset
1087 parent, filename = _path_split(path)
64240
cf399de12b54 One of the joys of having test_multiprocessing occasionally execute after
Brett Cannon <bcannon@gmail.com>
parents: 62761
diff changeset
1088 path_parts = []
cf399de12b54 One of the joys of having test_multiprocessing occasionally execute after
Brett Cannon <bcannon@gmail.com>
parents: 62761
diff changeset
1089 # Figure out what directories are missing.
cf399de12b54 One of the joys of having test_multiprocessing occasionally execute after
Brett Cannon <bcannon@gmail.com>
parents: 62761
diff changeset
1090 while parent and not _path_isdir(parent):
76449
163360370ad2 Make path manipulation more robust for platforms with alternative path
Brett Cannon <brett@python.org>
parents: 76445
diff changeset
1091 parent, part = _path_split(parent)
64240
cf399de12b54 One of the joys of having test_multiprocessing occasionally execute after
Brett Cannon <bcannon@gmail.com>
parents: 62761
diff changeset
1092 path_parts.append(part)
cf399de12b54 One of the joys of having test_multiprocessing occasionally execute after
Brett Cannon <bcannon@gmail.com>
parents: 62761
diff changeset
1093 # Create needed directories.
cf399de12b54 One of the joys of having test_multiprocessing occasionally execute after
Brett Cannon <bcannon@gmail.com>
parents: 62761
diff changeset
1094 for part in reversed(path_parts):
cf399de12b54 One of the joys of having test_multiprocessing occasionally execute after
Brett Cannon <bcannon@gmail.com>
parents: 62761
diff changeset
1095 parent = _path_join(parent, part)
cf399de12b54 One of the joys of having test_multiprocessing occasionally execute after
Brett Cannon <bcannon@gmail.com>
parents: 62761
diff changeset
1096 try:
cf399de12b54 One of the joys of having test_multiprocessing occasionally execute after
Brett Cannon <bcannon@gmail.com>
parents: 62761
diff changeset
1097 _os.mkdir(parent)
73169
e4d44c2e8e81 Simplify and remove few dependencies on 'errno', thanks to PEP 3151.
Florent Xicluna <florent.xicluna@gmail.com>
parents: 73016
diff changeset
1098 except FileExistsError:
64240
cf399de12b54 One of the joys of having test_multiprocessing occasionally execute after
Brett Cannon <bcannon@gmail.com>
parents: 62761
diff changeset
1099 # Probably another Python process already created the dir.
73169
e4d44c2e8e81 Simplify and remove few dependencies on 'errno', thanks to PEP 3151.
Florent Xicluna <florent.xicluna@gmail.com>
parents: 73016
diff changeset
1100 continue
79741
53ce30b62de2 Issue #15833: don't raise an exception if importlib can't write byte-compiled
Trent Nelson <trent@trent.me>
parents: 79652
diff changeset
1101 except OSError as exc:
53ce30b62de2 Issue #15833: don't raise an exception if importlib can't write byte-compiled
Trent Nelson <trent@trent.me>
parents: 79652
diff changeset
1102 # Could be a permission error, read-only filesystem: just forget
53ce30b62de2 Issue #15833: don't raise an exception if importlib can't write byte-compiled
Trent Nelson <trent@trent.me>
parents: 79652
diff changeset
1103 # about writing the data.
53ce30b62de2 Issue #15833: don't raise an exception if importlib can't write byte-compiled
Trent Nelson <trent@trent.me>
parents: 79652
diff changeset
1104 _verbose_message('could not create {!r}: {!r}', parent, exc)
73169
e4d44c2e8e81 Simplify and remove few dependencies on 'errno', thanks to PEP 3151.
Florent Xicluna <florent.xicluna@gmail.com>
parents: 73016
diff changeset
1105 return
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
1106 try:
78738
0c661c5632e0 Issue #2051: Tweak last commit for this issue to pass in mode instead
Brett Cannon <brett@python.org>
parents: 78722
diff changeset
1107 _write_atomic(path, data, _mode)
76753
59b8a7fcb047 Properly mark names in importlib._bootstrap as private.
Brett Cannon <brett@python.org>
parents: 76586
diff changeset
1108 _verbose_message('created {!r}', path)
79741
53ce30b62de2 Issue #15833: don't raise an exception if importlib can't write byte-compiled
Trent Nelson <trent@trent.me>
parents: 79652
diff changeset
1109 except OSError as exc:
53ce30b62de2 Issue #15833: don't raise an exception if importlib can't write byte-compiled
Trent Nelson <trent@trent.me>
parents: 79652
diff changeset
1110 # Same as above: just don't write the bytecode.
53ce30b62de2 Issue #15833: don't raise an exception if importlib can't write byte-compiled
Trent Nelson <trent@trent.me>
parents: 79652
diff changeset
1111 _verbose_message('could not create {!r}: {!r}', path, exc)
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
1112
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
1113
76548
a2cf07135e4f Issue #14605: Rename _SourcelessFileLoader to SourcelessFileLoader.
Marc-Andre Lemburg <mal@egenix.com>
parents: 76547
diff changeset
1114 class SourcelessFileLoader(FileLoader, _LoaderBasics):
55355
dfaf6de2c8ce Implement the PEP 302 protocol for get_filename() as
Brett Cannon <bcannon@gmail.com>
parents: 52633
diff changeset
1115
62756
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib.
Brett Cannon <bcannon@gmail.com>
parents: 62587
diff changeset
1116 """Loader which handles sourceless file imports."""
52112
03080fa08dfe Implement get_source for importlib.abc.PyLoader using source_path and get_data.
Brett Cannon <bcannon@gmail.com>
parents: 52083
diff changeset
1117
84001
e873f2e67353 Issues #18088, 18089: Introduce
Brett Cannon <brett@python.org>
parents: 83999
diff changeset
1118 def init_module_attrs(self, module):
e873f2e67353 Issues #18088, 18089: Introduce
Brett Cannon <brett@python.org>
parents: 83999
diff changeset
1119 super().init_module_attrs(module)
e873f2e67353 Issues #18088, 18089: Introduce
Brett Cannon <brett@python.org>
parents: 83999
diff changeset
1120 module.__cached__ = module.__file__
51771
ee7029710c05 Refactor source and bytecode file loaders in importlib so that there
Brett Cannon <bcannon@gmail.com>
parents: 51770
diff changeset
1121
ee7029710c05 Refactor source and bytecode file loaders in importlib so that there
Brett Cannon <bcannon@gmail.com>
parents: 51770
diff changeset
1122 def get_code(self, fullname):
62756
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib.
Brett Cannon <bcannon@gmail.com>
parents: 62587
diff changeset
1123 path = self.get_filename(fullname)
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib.
Brett Cannon <bcannon@gmail.com>
parents: 62587
diff changeset
1124 data = self.get_data(path)
81432
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
1125 bytes_data = _validate_bytecode_header(data, name=fullname, path=path)
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating
Brett Cannon <brett@python.org>
parents: 81428
diff changeset
1126 return _compile_bytecode(bytes_data, name=fullname, bytecode_path=path)
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
1127
62756
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib.
Brett Cannon <bcannon@gmail.com>
parents: 62587
diff changeset
1128 def get_source(self, fullname):
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib.
Brett Cannon <bcannon@gmail.com>
parents: 62587
diff changeset
1129 """Return None as there is no source code."""
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib.
Brett Cannon <bcannon@gmail.com>
parents: 62587
diff changeset
1130 return None
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
1131
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
1132
78486
1db6553f3f8c Issue #15576: Allow extension modules to be a package's __init__
Brett Cannon <brett@python.org>
parents: 78485
diff changeset
1133 # Filled in by _setup().
1db6553f3f8c Issue #15576: Allow extension modules to be a package's __init__
Brett Cannon <brett@python.org>
parents: 78485
diff changeset
1134 EXTENSION_SUFFIXES = []
1db6553f3f8c Issue #15576: Allow extension modules to be a package's __init__
Brett Cannon <brett@python.org>
parents: 78485
diff changeset
1135
1db6553f3f8c Issue #15576: Allow extension modules to be a package's __init__
Brett Cannon <brett@python.org>
parents: 78485
diff changeset
1136
76478
1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and
Brett Cannon <brett@python.org>
parents: 76472
diff changeset
1137 class ExtensionFileLoader:
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
1138
52141
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now.
Brett Cannon <bcannon@gmail.com>
parents: 52140
diff changeset
1139 """Loader for extension modules.
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
1140
52141
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now.
Brett Cannon <bcannon@gmail.com>
parents: 52140
diff changeset
1141 The constructor is designed to work with FileFinder.
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
1142
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
1143 """
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
1144
62756
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib.
Brett Cannon <bcannon@gmail.com>
parents: 62587
diff changeset
1145 def __init__(self, name, path):
76478
1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and
Brett Cannon <brett@python.org>
parents: 76472
diff changeset
1146 self.name = name
1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and
Brett Cannon <brett@python.org>
parents: 76472
diff changeset
1147 self.path = path
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
1148
52141
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now.
Brett Cannon <bcannon@gmail.com>
parents: 52140
diff changeset
1149 @_check_name
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now.
Brett Cannon <bcannon@gmail.com>
parents: 52140
diff changeset
1150 @set_package
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now.
Brett Cannon <bcannon@gmail.com>
parents: 52140
diff changeset
1151 @set_loader
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now.
Brett Cannon <bcannon@gmail.com>
parents: 52140
diff changeset
1152 def load_module(self, fullname):
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now.
Brett Cannon <bcannon@gmail.com>
parents: 52140
diff changeset
1153 """Load an extension module."""
84171
205aa49e5cd5 Issue #18115: Abstract out managing the cleanup of modules to use in
Brett Cannon <brett@python.org>
parents: 84169
diff changeset
1154 with _ManageReload(fullname):
78344
62033490ca0f Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise
Nick Coghlan <ncoghlan@gmail.com>
parents: 78314
diff changeset
1155 module = _call_with_frames_removed(_imp.load_dynamic,
62033490ca0f Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise
Nick Coghlan <ncoghlan@gmail.com>
parents: 78314
diff changeset
1156 fullname, self.path)
76753
59b8a7fcb047 Properly mark names in importlib._bootstrap as private.
Brett Cannon <brett@python.org>
parents: 76586
diff changeset
1157 _verbose_message('extension module loaded from {!r}', self.path)
78511
34d5ec8a1019 Don't overwrite a __path__ value from extension modules if already
Brett Cannon <brett@python.org>
parents: 78491
diff changeset
1158 if self.is_package(fullname) and not hasattr(module, '__path__'):
78486
1db6553f3f8c Issue #15576: Allow extension modules to be a package's __init__
Brett Cannon <brett@python.org>
parents: 78485
diff changeset
1159 module.__path__ = [_path_split(self.path)[0]]
76303
2dd046be2c88 Issue #2377: Make importlib the implementation of __import__().
Brett Cannon <brett@python.org>
parents: 76280
diff changeset
1160 return module
52141
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now.
Brett Cannon <bcannon@gmail.com>
parents: 52140
diff changeset
1161
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now.
Brett Cannon <bcannon@gmail.com>
parents: 52140
diff changeset
1162 def is_package(self, fullname):
78491
f7b59e890e30 fix docstring wording
Philip Jenvey <pjenvey@underboss.org>
parents: 78489
diff changeset
1163 """Return True if the extension module is a package."""
78486
1db6553f3f8c Issue #15576: Allow extension modules to be a package's __init__
Brett Cannon <brett@python.org>
parents: 78485
diff changeset
1164 file_name = _path_split(self.path)[1]
78487
e024f6ba5ed8 update docstring per the extension package fix, refactor
Philip Jenvey <pjenvey@underboss.org>
parents: 78486
diff changeset
1165 return any(file_name == '__init__' + suffix
e024f6ba5ed8 update docstring per the extension package fix, refactor
Philip Jenvey <pjenvey@underboss.org>
parents: 78486
diff changeset
1166 for suffix in EXTENSION_SUFFIXES)
52141
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now.
Brett Cannon <bcannon@gmail.com>
parents: 52140
diff changeset
1167
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now.
Brett Cannon <bcannon@gmail.com>
parents: 52140
diff changeset
1168 def get_code(self, fullname):
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now.
Brett Cannon <bcannon@gmail.com>
parents: 52140
diff changeset
1169 """Return None as an extension module cannot create a code object."""
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now.
Brett Cannon <bcannon@gmail.com>
parents: 52140
diff changeset
1170 return None
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now.
Brett Cannon <bcannon@gmail.com>
parents: 52140
diff changeset
1171
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now.
Brett Cannon <bcannon@gmail.com>
parents: 52140
diff changeset
1172 def get_source(self, fullname):
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now.
Brett Cannon <bcannon@gmail.com>
parents: 52140
diff changeset
1173 """Return None as extension modules have no source code."""
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now.
Brett Cannon <bcannon@gmail.com>
parents: 52140
diff changeset
1174 return None
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
1175
85946
832579dbafd6 [issue19152] Add ExtensionFileLoader.get_filename().
Eric Snow <ericsnowcurrently@gmail.com>
parents: 85945
diff changeset
1176 @_check_name
832579dbafd6 [issue19152] Add ExtensionFileLoader.get_filename().
Eric Snow <ericsnowcurrently@gmail.com>
parents: 85945
diff changeset
1177 def get_filename(self, fullname):
832579dbafd6 [issue19152] Add ExtensionFileLoader.get_filename().
Eric Snow <ericsnowcurrently@gmail.com>
parents: 85945
diff changeset
1178 """Return the path to the source file as found by the finder."""
832579dbafd6 [issue19152] Add ExtensionFileLoader.get_filename().
Eric Snow <ericsnowcurrently@gmail.com>
parents: 85945
diff changeset
1179 return self.path
832579dbafd6 [issue19152] Add ExtensionFileLoader.get_filename().
Eric Snow <ericsnowcurrently@gmail.com>
parents: 85945
diff changeset
1180
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
1181
77126
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1182 class _NamespacePath:
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1183 """Represents a namespace package's path. It uses the module name
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1184 to find its parent module, and from there it looks up the parent's
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1185 __path__. When this changes, the module's own path is recomputed,
80409
58d6fea0c0c7 missing letter
Benjamin Peterson <benjamin@python.org>
parents: 79832
diff changeset
1186 using path_finder. For top-level modules, the parent module's path
77126
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1187 is sys.path."""
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1188
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1189 def __init__(self, name, path, path_finder):
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1190 self._name = name
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1191 self._path = path
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1192 self._last_parent_path = tuple(self._get_parent_path())
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1193 self._path_finder = path_finder
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1194
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1195 def _find_parent_path_names(self):
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1196 """Returns a tuple of (parent-module-name, parent-path-attr-name)"""
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1197 parent, dot, me = self._name.rpartition('.')
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1198 if dot == '':
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1199 # This is a top-level module. sys.path contains the parent path.
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1200 return 'sys', 'path'
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1201 # Not a top-level module. parent-module.__path__ contains the
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1202 # parent path.
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1203 return parent, '__path__'
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1204
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1205 def _get_parent_path(self):
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1206 parent_module_name, path_attr_name = self._find_parent_path_names()
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1207 return getattr(sys.modules[parent_module_name], path_attr_name)
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1208
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1209 def _recalculate(self):
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1210 # If the parent's path has changed, recalculate _path
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1211 parent_path = tuple(self._get_parent_path()) # Make a copy
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1212 if parent_path != self._last_parent_path:
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1213 loader, new_path = self._path_finder(self._name, parent_path)
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1214 # Note that no changes are made if a loader is returned, but we
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1215 # do remember the new parent path
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1216 if loader is None:
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1217 self._path = new_path
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1218 self._last_parent_path = parent_path # Save the copy
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1219 return self._path
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1220
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1221 def __iter__(self):
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1222 return iter(self._recalculate())
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1223
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1224 def __len__(self):
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1225 return len(self._recalculate())
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1226
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1227 def __repr__(self):
84582
33d379c2bb79 Issue #18415: Normalize what type of quotes are used with string
Brett Cannon <brett@python.org>
parents: 84468
diff changeset
1228 return '_NamespacePath({!r})'.format(self._path)
77126
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1229
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1230 def __contains__(self, item):
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1231 return item in self._recalculate()
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1232
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1233 def append(self, item):
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1234 self._path.append(item)
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1235
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1236
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1237 class NamespaceLoader:
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1238 def __init__(self, name, path, path_finder):
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1239 self._path = _NamespacePath(name, path, path_finder)
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1240
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1241 @classmethod
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1242 def module_repr(cls, module):
84582
33d379c2bb79 Issue #18415: Normalize what type of quotes are used with string
Brett Cannon <brett@python.org>
parents: 84468
diff changeset
1243 return '<module {!r} (namespace)>'.format(module.__name__)
77126
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1244
84169
ebec625b13f9 Issues #18058, 18057: Make importlib._bootstrap.NamespaceLoader
Brett Cannon <brett@python.org>
parents: 84134
diff changeset
1245 def is_package(self, fullname):
ebec625b13f9 Issues #18058, 18057: Make importlib._bootstrap.NamespaceLoader
Brett Cannon <brett@python.org>
parents: 84134
diff changeset
1246 return True
ebec625b13f9 Issues #18058, 18057: Make importlib._bootstrap.NamespaceLoader
Brett Cannon <brett@python.org>
parents: 84134
diff changeset
1247
ebec625b13f9 Issues #18058, 18057: Make importlib._bootstrap.NamespaceLoader
Brett Cannon <brett@python.org>
parents: 84134
diff changeset
1248 def get_source(self, fullname):
ebec625b13f9 Issues #18058, 18057: Make importlib._bootstrap.NamespaceLoader
Brett Cannon <brett@python.org>
parents: 84134
diff changeset
1249 return ''
77126
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1250
84169
ebec625b13f9 Issues #18058, 18057: Make importlib._bootstrap.NamespaceLoader
Brett Cannon <brett@python.org>
parents: 84134
diff changeset
1251 def get_code(self, fullname):
ebec625b13f9 Issues #18058, 18057: Make importlib._bootstrap.NamespaceLoader
Brett Cannon <brett@python.org>
parents: 84134
diff changeset
1252 return compile('', '<string>', 'exec', dont_inherit=True)
ebec625b13f9 Issues #18058, 18057: Make importlib._bootstrap.NamespaceLoader
Brett Cannon <brett@python.org>
parents: 84134
diff changeset
1253
ebec625b13f9 Issues #18058, 18057: Make importlib._bootstrap.NamespaceLoader
Brett Cannon <brett@python.org>
parents: 84134
diff changeset
1254 def init_module_attrs(self, module):
ebec625b13f9 Issues #18058, 18057: Make importlib._bootstrap.NamespaceLoader
Brett Cannon <brett@python.org>
parents: 84134
diff changeset
1255 module.__loader__ = self
ebec625b13f9 Issues #18058, 18057: Make importlib._bootstrap.NamespaceLoader
Brett Cannon <brett@python.org>
parents: 84134
diff changeset
1256 module.__package__ = module.__name__
ebec625b13f9 Issues #18058, 18057: Make importlib._bootstrap.NamespaceLoader
Brett Cannon <brett@python.org>
parents: 84134
diff changeset
1257
84001
e873f2e67353 Issues #18088, 18089: Introduce
Brett Cannon <brett@python.org>
parents: 83999
diff changeset
1258 def load_module(self, fullname):
77126
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1259 """Load a namespace module."""
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1260 _verbose_message('namespace module loaded with path {!r}', self._path)
84001
e873f2e67353 Issues #18088, 18089: Introduce
Brett Cannon <brett@python.org>
parents: 83999
diff changeset
1261 with module_to_load(fullname) as module:
84169
ebec625b13f9 Issues #18058, 18057: Make importlib._bootstrap.NamespaceLoader
Brett Cannon <brett@python.org>
parents: 84134
diff changeset
1262 self.init_module_attrs(module)
84001
e873f2e67353 Issues #18088, 18089: Introduce
Brett Cannon <brett@python.org>
parents: 83999
diff changeset
1263 module.__path__ = self._path
e873f2e67353 Issues #18088, 18089: Introduce
Brett Cannon <brett@python.org>
parents: 83999
diff changeset
1264 return module
77126
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1265
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1266
52141
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now.
Brett Cannon <bcannon@gmail.com>
parents: 52140
diff changeset
1267 # Finders #####################################################################
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
1268
51322
53d6445c6c5a Rename importlib._bootstrap.SysPathFinder to PathFinder and expose off of
Brett Cannon <bcannon@gmail.com>
parents: 51266
diff changeset
1269 class PathFinder:
51266
63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to
Brett Cannon <bcannon@gmail.com>
parents: 51242
diff changeset
1270
78382
1f8351cf00f3 Issue #15502: Bring the importlib.PathFinder docs and docstring more in line with the new import system documentation, and fix various parts of the new docs that weren't quite right given PEP 420 or were otherwise a bit misleading. Also note the key terminology problem still being discussed in the issue
Nick Coghlan <ncoghlan@gmail.com>
parents: 78380
diff changeset
1271 """Meta path finder for sys.path and package __path__ attributes."""
51266
63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to
Brett Cannon <bcannon@gmail.com>
parents: 51242
diff changeset
1272
51639
ef2cba80c975 Change importlib.machinery.PathFinder to not have implicit semantics (that's
Brett Cannon <bcannon@gmail.com>
parents: 51433
diff changeset
1273 @classmethod
78485
0a75ce232f56 Issue #15502: Finish bringing importlib.abc in line with the current
Brett Cannon <brett@python.org>
parents: 78456
diff changeset
1274 def invalidate_caches(cls):
0a75ce232f56 Issue #15502: Finish bringing importlib.abc in line with the current
Brett Cannon <brett@python.org>
parents: 78456
diff changeset
1275 """Call the invalidate_caches() method on all path entry finders
0a75ce232f56 Issue #15502: Finish bringing importlib.abc in line with the current
Brett Cannon <brett@python.org>
parents: 78456
diff changeset
1276 stored in sys.path_importer_caches (where implemented)."""
0a75ce232f56 Issue #15502: Finish bringing importlib.abc in line with the current
Brett Cannon <brett@python.org>
parents: 78456
diff changeset
1277 for finder in sys.path_importer_cache.values():
0a75ce232f56 Issue #15502: Finish bringing importlib.abc in line with the current
Brett Cannon <brett@python.org>
parents: 78456
diff changeset
1278 if hasattr(finder, 'invalidate_caches'):
0a75ce232f56 Issue #15502: Finish bringing importlib.abc in line with the current
Brett Cannon <brett@python.org>
parents: 78456
diff changeset
1279 finder.invalidate_caches()
0a75ce232f56 Issue #15502: Finish bringing importlib.abc in line with the current
Brett Cannon <brett@python.org>
parents: 78456
diff changeset
1280
0a75ce232f56 Issue #15502: Finish bringing importlib.abc in line with the current
Brett Cannon <brett@python.org>
parents: 78456
diff changeset
1281 @classmethod
76566
57d558f1904d Issue #14605: Make explicit the entries on sys.path_hooks that used to
Brett Cannon <brett@python.org>
parents: 76549
diff changeset
1282 def _path_hooks(cls, path):
51639
ef2cba80c975 Change importlib.machinery.PathFinder to not have implicit semantics (that's
Brett Cannon <bcannon@gmail.com>
parents: 51433
diff changeset
1283 """Search sequence of hooks for a finder for 'path'.
51266
63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to
Brett Cannon <bcannon@gmail.com>
parents: 51242
diff changeset
1284
51639
ef2cba80c975 Change importlib.machinery.PathFinder to not have implicit semantics (that's
Brett Cannon <bcannon@gmail.com>
parents: 51433
diff changeset
1285 If 'hooks' is false then use sys.path_hooks.
51266
63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to
Brett Cannon <bcannon@gmail.com>
parents: 51242
diff changeset
1286
63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to
Brett Cannon <bcannon@gmail.com>
parents: 51242
diff changeset
1287 """
76566
57d558f1904d Issue #14605: Make explicit the entries on sys.path_hooks that used to
Brett Cannon <brett@python.org>
parents: 76549
diff changeset
1288 if not sys.path_hooks:
57d558f1904d Issue #14605: Make explicit the entries on sys.path_hooks that used to
Brett Cannon <brett@python.org>
parents: 76549
diff changeset
1289 _warnings.warn('sys.path_hooks is empty', ImportWarning)
57d558f1904d Issue #14605: Make explicit the entries on sys.path_hooks that used to
Brett Cannon <brett@python.org>
parents: 76549
diff changeset
1290 for hook in sys.path_hooks:
51266
63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to
Brett Cannon <bcannon@gmail.com>
parents: 51242
diff changeset
1291 try:
63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to
Brett Cannon <bcannon@gmail.com>
parents: 51242
diff changeset
1292 return hook(path)
63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to
Brett Cannon <bcannon@gmail.com>
parents: 51242
diff changeset
1293 except ImportError:
63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to
Brett Cannon <bcannon@gmail.com>
parents: 51242
diff changeset
1294 continue
63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to
Brett Cannon <bcannon@gmail.com>
parents: 51242
diff changeset
1295 else:
76582
7025ee00dbf6 Issue #14605: Use None in sys.path_importer_cache to represent no
Brett Cannon <brett@python.org>
parents: 76579
diff changeset
1296 return None
51266
63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to
Brett Cannon <bcannon@gmail.com>
parents: 51242
diff changeset
1297
51322
53d6445c6c5a Rename importlib._bootstrap.SysPathFinder to PathFinder and expose off of
Brett Cannon <bcannon@gmail.com>
parents: 51266
diff changeset
1298 @classmethod
76566
57d558f1904d Issue #14605: Make explicit the entries on sys.path_hooks that used to
Brett Cannon <brett@python.org>
parents: 76549
diff changeset
1299 def _path_importer_cache(cls, path):
78382
1f8351cf00f3 Issue #15502: Bring the importlib.PathFinder docs and docstring more in line with the new import system documentation, and fix various parts of the new docs that weren't quite right given PEP 420 or were otherwise a bit misleading. Also note the key terminology problem still being discussed in the issue
Nick Coghlan <ncoghlan@gmail.com>
parents: 78380
diff changeset
1300 """Get the finder for the path entry from sys.path_importer_cache.
51266
63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to
Brett Cannon <bcannon@gmail.com>
parents: 51242
diff changeset
1301
78382
1f8351cf00f3 Issue #15502: Bring the importlib.PathFinder docs and docstring more in line with the new import system documentation, and fix various parts of the new docs that weren't quite right given PEP 420 or were otherwise a bit misleading. Also note the key terminology problem still being discussed in the issue
Nick Coghlan <ncoghlan@gmail.com>
parents: 78380
diff changeset
1302 If the path entry is not in the cache, find the appropriate finder
1f8351cf00f3 Issue #15502: Bring the importlib.PathFinder docs and docstring more in line with the new import system documentation, and fix various parts of the new docs that weren't quite right given PEP 420 or were otherwise a bit misleading. Also note the key terminology problem still being discussed in the issue
Nick Coghlan <ncoghlan@gmail.com>
parents: 78380
diff changeset
1303 and cache it. If no finder is available, store None.
51266
63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to
Brett Cannon <bcannon@gmail.com>
parents: 51242
diff changeset
1304
63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to
Brett Cannon <bcannon@gmail.com>
parents: 51242
diff changeset
1305 """
74994
b8593ec7e8c5 Tweak the handling of the empty string in sys.path for importlib.
Brett Cannon <brett@python.org>
parents: 74992
diff changeset
1306 if path == '':
86436
76184b5339f2 Issue #18416: Have importlib.machinery.PathFinder treat '' as the cwd
Brett Cannon <brett@python.org>
parents: 85952
diff changeset
1307 path = _os.getcwd()
51266
63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to
Brett Cannon <bcannon@gmail.com>
parents: 51242
diff changeset
1308 try:
51322
53d6445c6c5a Rename importlib._bootstrap.SysPathFinder to PathFinder and expose off of
Brett Cannon <bcannon@gmail.com>
parents: 51266
diff changeset
1309 finder = sys.path_importer_cache[path]
51266
63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to
Brett Cannon <bcannon@gmail.com>
parents: 51242
diff changeset
1310 except KeyError:
51322
53d6445c6c5a Rename importlib._bootstrap.SysPathFinder to PathFinder and expose off of
Brett Cannon <bcannon@gmail.com>
parents: 51266
diff changeset
1311 finder = cls._path_hooks(path)
51266
63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to
Brett Cannon <bcannon@gmail.com>
parents: 51242
diff changeset
1312 sys.path_importer_cache[path] = finder
63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to
Brett Cannon <bcannon@gmail.com>
parents: 51242
diff changeset
1313 return finder
63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to
Brett Cannon <bcannon@gmail.com>
parents: 51242
diff changeset
1314
51322
53d6445c6c5a Rename importlib._bootstrap.SysPathFinder to PathFinder and expose off of
Brett Cannon <bcannon@gmail.com>
parents: 51266
diff changeset
1315 @classmethod
77126
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1316 def _get_loader(cls, fullname, path):
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1317 """Find the loader or namespace_path for this module/package name."""
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1318 # If this ends up being a namespace package, namespace_path is
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1319 # the list of paths that will become its __path__
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1320 namespace_path = []
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1321 for entry in path:
80529
291406748217 - Issue #16514: Fix regression causing a traceback when sys.path[0] is None
Barry Warsaw <barry@python.org>
parents: 80409
diff changeset
1322 if not isinstance(entry, (str, bytes)):
291406748217 - Issue #16514: Fix regression causing a traceback when sys.path[0] is None
Barry Warsaw <barry@python.org>
parents: 80409
diff changeset
1323 continue
77126
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1324 finder = cls._path_importer_cache(entry)
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1325 if finder is not None:
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1326 if hasattr(finder, 'find_loader'):
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1327 loader, portions = finder.find_loader(fullname)
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1328 else:
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1329 loader = finder.find_module(fullname)
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1330 portions = []
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1331 if loader is not None:
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1332 # We found a loader: return it immediately.
78485
0a75ce232f56 Issue #15502: Finish bringing importlib.abc in line with the current
Brett Cannon <brett@python.org>
parents: 78456
diff changeset
1333 return loader, namespace_path
77126
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1334 # This is possibly part of a namespace package.
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1335 # Remember these path entries (if any) for when we
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1336 # create a namespace package, and continue iterating
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1337 # on path.
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1338 namespace_path.extend(portions)
51266
63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to
Brett Cannon <bcannon@gmail.com>
parents: 51242
diff changeset
1339 else:
78485
0a75ce232f56 Issue #15502: Finish bringing importlib.abc in line with the current
Brett Cannon <brett@python.org>
parents: 78456
diff changeset
1340 return None, namespace_path
51266
63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to
Brett Cannon <bcannon@gmail.com>
parents: 51242
diff changeset
1341
51322
53d6445c6c5a Rename importlib._bootstrap.SysPathFinder to PathFinder and expose off of
Brett Cannon <bcannon@gmail.com>
parents: 51266
diff changeset
1342 @classmethod
53d6445c6c5a Rename importlib._bootstrap.SysPathFinder to PathFinder and expose off of
Brett Cannon <bcannon@gmail.com>
parents: 51266
diff changeset
1343 def find_module(cls, fullname, path=None):
52165
62f9dad628de A few more docstring/API cleanups for importlib.
Brett Cannon <bcannon@gmail.com>
parents: 52164
diff changeset
1344 """Find the module on sys.path or 'path' based on sys.path_hooks and
62f9dad628de A few more docstring/API cleanups for importlib.
Brett Cannon <bcannon@gmail.com>
parents: 52164
diff changeset
1345 sys.path_importer_cache."""
76384
c1399cf7bd6a Issue #12599: Be more strict in accepting None vs. a false-like object
Brett Cannon <brett@python.org>
parents: 76331
diff changeset
1346 if path is None:
51266
63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to
Brett Cannon <bcannon@gmail.com>
parents: 51242
diff changeset
1347 path = sys.path
77126
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1348 loader, namespace_path = cls._get_loader(fullname, path)
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1349 if loader is not None:
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1350 return loader
51266
63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to
Brett Cannon <bcannon@gmail.com>
parents: 51242
diff changeset
1351 else:
77126
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1352 if namespace_path:
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1353 # We found at least one namespace path. Return a
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1354 # loader which can create the namespace package.
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1355 return NamespaceLoader(fullname, namespace_path, cls._get_loader)
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1356 else:
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1357 return None
51266
63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to
Brett Cannon <bcannon@gmail.com>
parents: 51242
diff changeset
1358
63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to
Brett Cannon <bcannon@gmail.com>
parents: 51242
diff changeset
1359
76478
1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and
Brett Cannon <brett@python.org>
parents: 76472
diff changeset
1360 class FileFinder:
52141
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now.
Brett Cannon <bcannon@gmail.com>
parents: 52140
diff changeset
1361
62756
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib.
Brett Cannon <bcannon@gmail.com>
parents: 62587
diff changeset
1362 """File-based finder.
52141
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now.
Brett Cannon <bcannon@gmail.com>
parents: 52140
diff changeset
1363
76478
1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and
Brett Cannon <brett@python.org>
parents: 76472
diff changeset
1364 Interactions with the file system are cached for performance, being
1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and
Brett Cannon <brett@python.org>
parents: 76472
diff changeset
1365 refreshed when the directory the finder is handling has been modified.
52141
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now.
Brett Cannon <bcannon@gmail.com>
parents: 52140
diff changeset
1366
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now.
Brett Cannon <bcannon@gmail.com>
parents: 52140
diff changeset
1367 """
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now.
Brett Cannon <bcannon@gmail.com>
parents: 52140
diff changeset
1368
84174
e13371d5b4e4 Fix a misnaming of a method and an argument
Brett Cannon <brett@python.org>
parents: 83055
diff changeset
1369 def __init__(self, path, *loader_details):
76478
1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and
Brett Cannon <brett@python.org>
parents: 76472
diff changeset
1370 """Initialize with the path to search on and a variable number of
82236
0f65bf6063ca Fixes a FileFinder docstring to reflect an old change.
Eric Snow <ericsnowcurrently@gmail.com>
parents: 81884
diff changeset
1371 2-tuples containing the loader and the file suffixes the loader
0f65bf6063ca Fixes a FileFinder docstring to reflect an old change.
Eric Snow <ericsnowcurrently@gmail.com>
parents: 81884
diff changeset
1372 recognizes."""
78486
1db6553f3f8c Issue #15576: Allow extension modules to be a package's __init__
Brett Cannon <brett@python.org>
parents: 78485
diff changeset
1373 loaders = []
84174
e13371d5b4e4 Fix a misnaming of a method and an argument
Brett Cannon <brett@python.org>
parents: 83055
diff changeset
1374 for loader, suffixes in loader_details:
78486
1db6553f3f8c Issue #15576: Allow extension modules to be a package's __init__
Brett Cannon <brett@python.org>
parents: 78485
diff changeset
1375 loaders.extend((suffix, loader) for suffix in suffixes)
1db6553f3f8c Issue #15576: Allow extension modules to be a package's __init__
Brett Cannon <brett@python.org>
parents: 78485
diff changeset
1376 self._loaders = loaders
75048
bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function.
Antoine Pitrou <solipsis@pitrou.net>
parents: 75045
diff changeset
1377 # Base (directory) path
86436
76184b5339f2 Issue #18416: Have importlib.machinery.PathFinder treat '' as the cwd
Brett Cannon <brett@python.org>
parents: 85952
diff changeset
1378 self.path = path
75048
bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function.
Antoine Pitrou <solipsis@pitrou.net>
parents: 75045
diff changeset
1379 self._path_mtime = -1
bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function.
Antoine Pitrou <solipsis@pitrou.net>
parents: 75045
diff changeset
1380 self._path_cache = set()
75082
3297dcdad196 Issue #14063: fix test_importlib failure under OS X case-insensitive filesystems
Antoine Pitrou <solipsis@pitrou.net>
parents: 75070
diff changeset
1381 self._relaxed_path_cache = set()
52141
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now.
Brett Cannon <bcannon@gmail.com>
parents: 52140
diff changeset
1382
75324
de125ee32897 Update importlib.invalidate_caches() to be more general.
Brett Cannon <brett@python.org>
parents: 75250
diff changeset
1383 def invalidate_caches(self):
de125ee32897 Update importlib.invalidate_caches() to be more general.
Brett Cannon <brett@python.org>
parents: 75250
diff changeset
1384 """Invalidate the directory mtime."""
de125ee32897 Update importlib.invalidate_caches() to be more general.
Brett Cannon <brett@python.org>
parents: 75250
diff changeset
1385 self._path_mtime = -1
52141
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now.
Brett Cannon <bcannon@gmail.com>
parents: 52140
diff changeset
1386
78489
e7a67f1bf604 Issue #15502: Refactor some code.
Brett Cannon <brett@python.org>
parents: 78487
diff changeset
1387 find_module = _find_module_shim
77126
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1388
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1389 def find_loader(self, fullname):
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1390 """Try to find a loader for the specified module, or the namespace
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1391 package portions. Returns (loader, list-of-portions)."""
77757
2051fead1933 Fixes issue 15039: namespace packages are no longer imported in preference to modules of the same name.
Eric V. Smith <eric@trueblade.com>
parents: 77600
diff changeset
1392 is_namespace = False
52141
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now.
Brett Cannon <bcannon@gmail.com>
parents: 52140
diff changeset
1393 tail_module = fullname.rpartition('.')[2]
75048
bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function.
Antoine Pitrou <solipsis@pitrou.net>
parents: 75045
diff changeset
1394 try:
86438
33844153cd02 Issue #18416: Fix various os calls in importlib.machinery.FileFinder
Brett Cannon <brett@python.org>
parents: 86436
diff changeset
1395 mtime = _os.stat(self.path or _os.getcwd()).st_mtime
75048
bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function.
Antoine Pitrou <solipsis@pitrou.net>
parents: 75045
diff changeset
1396 except OSError:
bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function.
Antoine Pitrou <solipsis@pitrou.net>
parents: 75045
diff changeset
1397 mtime = -1
75324
de125ee32897 Update importlib.invalidate_caches() to be more general.
Brett Cannon <brett@python.org>
parents: 75250
diff changeset
1398 if mtime != self._path_mtime:
75048
bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function.
Antoine Pitrou <solipsis@pitrou.net>
parents: 75045
diff changeset
1399 self._fill_cache()
bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function.
Antoine Pitrou <solipsis@pitrou.net>
parents: 75045
diff changeset
1400 self._path_mtime = mtime
75082
3297dcdad196 Issue #14063: fix test_importlib failure under OS X case-insensitive filesystems
Antoine Pitrou <solipsis@pitrou.net>
parents: 75070
diff changeset
1401 # tail_module keeps the original casing, for __file__ and friends
3297dcdad196 Issue #14063: fix test_importlib failure under OS X case-insensitive filesystems
Antoine Pitrou <solipsis@pitrou.net>
parents: 75070
diff changeset
1402 if _relax_case():
3297dcdad196 Issue #14063: fix test_importlib failure under OS X case-insensitive filesystems
Antoine Pitrou <solipsis@pitrou.net>
parents: 75070
diff changeset
1403 cache = self._relaxed_path_cache
3297dcdad196 Issue #14063: fix test_importlib failure under OS X case-insensitive filesystems
Antoine Pitrou <solipsis@pitrou.net>
parents: 75070
diff changeset
1404 cache_module = tail_module.lower()
3297dcdad196 Issue #14063: fix test_importlib failure under OS X case-insensitive filesystems
Antoine Pitrou <solipsis@pitrou.net>
parents: 75070
diff changeset
1405 else:
3297dcdad196 Issue #14063: fix test_importlib failure under OS X case-insensitive filesystems
Antoine Pitrou <solipsis@pitrou.net>
parents: 75070
diff changeset
1406 cache = self._path_cache
3297dcdad196 Issue #14063: fix test_importlib failure under OS X case-insensitive filesystems
Antoine Pitrou <solipsis@pitrou.net>
parents: 75070
diff changeset
1407 cache_module = tail_module
76309
e2f48b076012 Add some comments.
Brett Cannon <brett@python.org>
parents: 76303
diff changeset
1408 # Check if the module is the name of a directory (and thus a package).
75082
3297dcdad196 Issue #14063: fix test_importlib failure under OS X case-insensitive filesystems
Antoine Pitrou <solipsis@pitrou.net>
parents: 75070
diff changeset
1409 if cache_module in cache:
75048
bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function.
Antoine Pitrou <solipsis@pitrou.net>
parents: 75045
diff changeset
1410 base_path = _path_join(self.path, tail_module)
86444
11f2f4af1979 Issue #18810: Be optimistic with stat calls when seeing if a directory
Brett Cannon <brett@python.org>
parents: 86438
diff changeset
1411 for suffix, loader in self._loaders:
11f2f4af1979 Issue #18810: Be optimistic with stat calls when seeing if a directory
Brett Cannon <brett@python.org>
parents: 86438
diff changeset
1412 init_filename = '__init__' + suffix
11f2f4af1979 Issue #18810: Be optimistic with stat calls when seeing if a directory
Brett Cannon <brett@python.org>
parents: 86438
diff changeset
1413 full_path = _path_join(base_path, init_filename)
11f2f4af1979 Issue #18810: Be optimistic with stat calls when seeing if a directory
Brett Cannon <brett@python.org>
parents: 86438
diff changeset
1414 if _path_isfile(full_path):
11f2f4af1979 Issue #18810: Be optimistic with stat calls when seeing if a directory
Brett Cannon <brett@python.org>
parents: 86438
diff changeset
1415 return (loader(fullname, full_path), [base_path])
11f2f4af1979 Issue #18810: Be optimistic with stat calls when seeing if a directory
Brett Cannon <brett@python.org>
parents: 86438
diff changeset
1416 else:
11f2f4af1979 Issue #18810: Be optimistic with stat calls when seeing if a directory
Brett Cannon <brett@python.org>
parents: 86438
diff changeset
1417 # If a namespace package, return the path if we don't
11f2f4af1979 Issue #18810: Be optimistic with stat calls when seeing if a directory
Brett Cannon <brett@python.org>
parents: 86438
diff changeset
1418 # find a module in the next section.
11f2f4af1979 Issue #18810: Be optimistic with stat calls when seeing if a directory
Brett Cannon <brett@python.org>
parents: 86438
diff changeset
1419 is_namespace = _path_isdir(base_path)
76309
e2f48b076012 Add some comments.
Brett Cannon <brett@python.org>
parents: 76303
diff changeset
1420 # Check for a file w/ a proper suffix exists.
78486
1db6553f3f8c Issue #15576: Allow extension modules to be a package's __init__
Brett Cannon <brett@python.org>
parents: 78485
diff changeset
1421 for suffix, loader in self._loaders:
83055
ac89e7bc0e2f Issue #17357: Add missing verbosity messages when running under
Brett Cannon <brett@python.org>
parents: 82236
diff changeset
1422 full_path = _path_join(self.path, tail_module + suffix)
ac89e7bc0e2f Issue #17357: Add missing verbosity messages when running under
Brett Cannon <brett@python.org>
parents: 82236
diff changeset
1423 _verbose_message('trying {}'.format(full_path), verbosity=2)
75082
3297dcdad196 Issue #14063: fix test_importlib failure under OS X case-insensitive filesystems
Antoine Pitrou <solipsis@pitrou.net>
parents: 75070
diff changeset
1424 if cache_module + suffix in cache:
75048
bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function.
Antoine Pitrou <solipsis@pitrou.net>
parents: 75045
diff changeset
1425 if _path_isfile(full_path):
77126
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1426 return (loader(fullname, full_path), [])
77757
2051fead1933 Fixes issue 15039: namespace packages are no longer imported in preference to modules of the same name.
Eric V. Smith <eric@trueblade.com>
parents: 77600
diff changeset
1427 if is_namespace:
83055
ac89e7bc0e2f Issue #17357: Add missing verbosity messages when running under
Brett Cannon <brett@python.org>
parents: 82236
diff changeset
1428 _verbose_message('possible namespace for {}'.format(base_path))
77757
2051fead1933 Fixes issue 15039: namespace packages are no longer imported in preference to modules of the same name.
Eric V. Smith <eric@trueblade.com>
parents: 77600
diff changeset
1429 return (None, [base_path])
77126
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages.
Eric V. Smith <eric@trueblade.com>
parents: 77013
diff changeset
1430 return (None, [])
62756
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib.
Brett Cannon <bcannon@gmail.com>
parents: 62587
diff changeset
1431
75048
bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function.
Antoine Pitrou <solipsis@pitrou.net>
parents: 75045
diff changeset
1432 def _fill_cache(self):
bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function.
Antoine Pitrou <solipsis@pitrou.net>
parents: 75045
diff changeset
1433 """Fill the cache of potential modules and packages for this directory."""
bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function.
Antoine Pitrou <solipsis@pitrou.net>
parents: 75045
diff changeset
1434 path = self.path
78663
bfd04bfb55c5 Close #14846: Handle a sys.path entry going away
Nick Coghlan <ncoghlan@gmail.com>
parents: 78619
diff changeset
1435 try:
86438
33844153cd02 Issue #18416: Fix various os calls in importlib.machinery.FileFinder
Brett Cannon <brett@python.org>
parents: 86436
diff changeset
1436 contents = _os.listdir(path or _os.getcwd())
81427
159967aa24a5 Issue #16730: Don't raise an exception in
Brett Cannon <brett@python.org>
parents: 80929
diff changeset
1437 except (FileNotFoundError, PermissionError, NotADirectoryError):
159967aa24a5 Issue #16730: Don't raise an exception in
Brett Cannon <brett@python.org>
parents: 80929
diff changeset
1438 # Directory has either been removed, turned into a file, or made
159967aa24a5 Issue #16730: Don't raise an exception in
Brett Cannon <brett@python.org>
parents: 80929
diff changeset
1439 # unreadable.
78663
bfd04bfb55c5 Close #14846: Handle a sys.path entry going away
Nick Coghlan <ncoghlan@gmail.com>
parents: 78619
diff changeset
1440 contents = []
75082
3297dcdad196 Issue #14063: fix test_importlib failure under OS X case-insensitive filesystems
Antoine Pitrou <solipsis@pitrou.net>
parents: 75070
diff changeset
1441 # We store two cached versions, to handle runtime changes of the
3297dcdad196 Issue #14063: fix test_importlib failure under OS X case-insensitive filesystems
Antoine Pitrou <solipsis@pitrou.net>
parents: 75070
diff changeset
1442 # PYTHONCASEOK environment variable.
76428
a32be109bd86 Issue #14581: Windows users are allowed to import modules w/o taking
Brett Cannon <brett@python.org>
parents: 76406
diff changeset
1443 if not sys.platform.startswith('win'):
a32be109bd86 Issue #14581: Windows users are allowed to import modules w/o taking
Brett Cannon <brett@python.org>
parents: 76406
diff changeset
1444 self._path_cache = set(contents)
a32be109bd86 Issue #14581: Windows users are allowed to import modules w/o taking
Brett Cannon <brett@python.org>
parents: 76406
diff changeset
1445 else:
a32be109bd86 Issue #14581: Windows users are allowed to import modules w/o taking
Brett Cannon <brett@python.org>
parents: 76406
diff changeset
1446 # Windows users can import modules with case-insensitive file
a32be109bd86 Issue #14581: Windows users are allowed to import modules w/o taking
Brett Cannon <brett@python.org>
parents: 76406
diff changeset
1447 # suffixes (for legacy reasons). Make the suffix lowercase here
a32be109bd86 Issue #14581: Windows users are allowed to import modules w/o taking
Brett Cannon <brett@python.org>
parents: 76406
diff changeset
1448 # so it's done once instead of for every import. This is safe as
a32be109bd86 Issue #14581: Windows users are allowed to import modules w/o taking
Brett Cannon <brett@python.org>
parents: 76406
diff changeset
1449 # the specified suffixes to check against are always specified in a
a32be109bd86 Issue #14581: Windows users are allowed to import modules w/o taking
Brett Cannon <brett@python.org>
parents: 76406
diff changeset
1450 # case-sensitive manner.
a32be109bd86 Issue #14581: Windows users are allowed to import modules w/o taking
Brett Cannon <brett@python.org>
parents: 76406
diff changeset
1451 lower_suffix_contents = set()
a32be109bd86 Issue #14581: Windows users are allowed to import modules w/o taking
Brett Cannon <brett@python.org>
parents: 76406
diff changeset
1452 for item in contents:
a32be109bd86 Issue #14581: Windows users are allowed to import modules w/o taking
Brett Cannon <brett@python.org>
parents: 76406
diff changeset
1453 name, dot, suffix = item.partition('.')
a32be109bd86 Issue #14581: Windows users are allowed to import modules w/o taking
Brett Cannon <brett@python.org>
parents: 76406
diff changeset
1454 if dot:
a32be109bd86 Issue #14581: Windows users are allowed to import modules w/o taking
Brett Cannon <brett@python.org>
parents: 76406
diff changeset
1455 new_name = '{}.{}'.format(name, suffix.lower())
a32be109bd86 Issue #14581: Windows users are allowed to import modules w/o taking
Brett Cannon <brett@python.org>
parents: 76406
diff changeset
1456 else:
a32be109bd86 Issue #14581: Windows users are allowed to import modules w/o taking
Brett Cannon <brett@python.org>
parents: 76406
diff changeset
1457 new_name = name
a32be109bd86 Issue #14581: Windows users are allowed to import modules w/o taking
Brett Cannon <brett@python.org>
parents: 76406
diff changeset
1458 lower_suffix_contents.add(new_name)
a32be109bd86 Issue #14581: Windows users are allowed to import modules w/o taking
Brett Cannon <brett@python.org>
parents: 76406
diff changeset
1459 self._path_cache = lower_suffix_contents
76753
59b8a7fcb047 Properly mark names in importlib._bootstrap as private.
Brett Cannon <brett@python.org>
parents: 76586
diff changeset
1460 if sys.platform.startswith(_CASE_INSENSITIVE_PLATFORMS):
82382
2528e4aea338 Issue #17220: Little cleanup of _bootstrap.py.
Serhiy Storchaka <storchaka@gmail.com>
parents: 82237
diff changeset
1461 self._relaxed_path_cache = {fn.lower() for fn in contents}
62756
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib.
Brett Cannon <bcannon@gmail.com>
parents: 62587
diff changeset
1462
76478
1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and
Brett Cannon <brett@python.org>
parents: 76472
diff changeset
1463 @classmethod
1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and
Brett Cannon <brett@python.org>
parents: 76472
diff changeset
1464 def path_hook(cls, *loader_details):
1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and
Brett Cannon <brett@python.org>
parents: 76472
diff changeset
1465 """A class method which returns a closure to use on sys.path_hook
1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and
Brett Cannon <brett@python.org>
parents: 76472
diff changeset
1466 which will return an instance using the specified loaders and the path
1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and
Brett Cannon <brett@python.org>
parents: 76472
diff changeset
1467 called on the closure.
52141
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now.
Brett Cannon <bcannon@gmail.com>
parents: 52140
diff changeset
1468
76478
1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and
Brett Cannon <brett@python.org>
parents: 76472
diff changeset
1469 If the path called on the closure is not a directory, ImportError is
1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and
Brett Cannon <brett@python.org>
parents: 76472
diff changeset
1470 raised.
52141
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now.
Brett Cannon <bcannon@gmail.com>
parents: 52140
diff changeset
1471
76478
1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and
Brett Cannon <brett@python.org>
parents: 76472
diff changeset
1472 """
1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and
Brett Cannon <brett@python.org>
parents: 76472
diff changeset
1473 def path_hook_for_FileFinder(path):
1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and
Brett Cannon <brett@python.org>
parents: 76472
diff changeset
1474 """Path hook for importlib.machinery.FileFinder."""
1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and
Brett Cannon <brett@python.org>
parents: 76472
diff changeset
1475 if not _path_isdir(path):
84582
33d379c2bb79 Issue #18415: Normalize what type of quotes are used with string
Brett Cannon <brett@python.org>
parents: 84468
diff changeset
1476 raise ImportError('only directories are supported', path=path)
76478
1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and
Brett Cannon <brett@python.org>
parents: 76472
diff changeset
1477 return cls(path, *loader_details)
52141
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now.
Brett Cannon <bcannon@gmail.com>
parents: 52140
diff changeset
1478
76478
1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and
Brett Cannon <brett@python.org>
parents: 76472
diff changeset
1479 return path_hook_for_FileFinder
52141
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now.
Brett Cannon <bcannon@gmail.com>
parents: 52140
diff changeset
1480
77600
df856963fc6c A better repr() for FileFinder
Antoine Pitrou <solipsis@pitrou.net>
parents: 77457
diff changeset
1481 def __repr__(self):
84582
33d379c2bb79 Issue #18415: Normalize what type of quotes are used with string
Brett Cannon <brett@python.org>
parents: 84468
diff changeset
1482 return 'FileFinder({!r})'.format(self.path)
52141
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now.
Brett Cannon <bcannon@gmail.com>
parents: 52140
diff changeset
1483
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now.
Brett Cannon <bcannon@gmail.com>
parents: 52140
diff changeset
1484
52140
1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap.
Brett Cannon <bcannon@gmail.com>
parents: 52139
diff changeset
1485 # Import itself ###############################################################
1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap.
Brett Cannon <bcannon@gmail.com>
parents: 52139
diff changeset
1486
52141
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now.
Brett Cannon <bcannon@gmail.com>
parents: 52140
diff changeset
1487 class _ImportLockContext:
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now.
Brett Cannon <bcannon@gmail.com>
parents: 52140
diff changeset
1488
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now.
Brett Cannon <bcannon@gmail.com>
parents: 52140
diff changeset
1489 """Context manager for the import lock."""
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now.
Brett Cannon <bcannon@gmail.com>
parents: 52140
diff changeset
1490
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now.
Brett Cannon <bcannon@gmail.com>
parents: 52140
diff changeset
1491 def __enter__(self):
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now.
Brett Cannon <bcannon@gmail.com>
parents: 52140
diff changeset
1492 """Acquire the import lock."""
76331
d777f854a66e Issue #13959: Rename imp to _imp and add Lib/imp.py and begin
Brett Cannon <brett@python.org>
parents: 76330
diff changeset
1493 _imp.acquire_lock()
52141
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now.
Brett Cannon <bcannon@gmail.com>
parents: 52140
diff changeset
1494
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now.
Brett Cannon <bcannon@gmail.com>
parents: 52140
diff changeset
1495 def __exit__(self, exc_type, exc_value, exc_traceback):
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now.
Brett Cannon <bcannon@gmail.com>
parents: 52140
diff changeset
1496 """Release the import lock regardless of any raised exceptions."""
76331
d777f854a66e Issue #13959: Rename imp to _imp and add Lib/imp.py and begin
Brett Cannon <brett@python.org>
parents: 76330
diff changeset
1497 _imp.release_lock()
52141
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now.
Brett Cannon <bcannon@gmail.com>
parents: 52140
diff changeset
1498
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
1499
74983
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using
Brett Cannon <brett@python.org>
parents: 74935
diff changeset
1500 def _resolve_name(name, package, level):
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using
Brett Cannon <brett@python.org>
parents: 74935
diff changeset
1501 """Resolve a relative module name to an absolute one."""
75250
8840037a9c53 unused imports, pep8
Philip Jenvey <pjenvey@underboss.org>
parents: 75239
diff changeset
1502 bits = package.rsplit('.', level - 1)
75239
5cfc9c97af23 Simplify importib._resolve_name().
Brett Cannon <brett@python.org>
parents: 75235
diff changeset
1503 if len(bits) < level:
5cfc9c97af23 Simplify importib._resolve_name().
Brett Cannon <brett@python.org>
parents: 75235
diff changeset
1504 raise ValueError('attempted relative import beyond top-level package')
5cfc9c97af23 Simplify importib._resolve_name().
Brett Cannon <brett@python.org>
parents: 75235
diff changeset
1505 base = bits[0]
77964
37850f083af5 Minor refactoring in importlib._bootstrap, and fix the '_wrap' docstring.
Florent Xicluna <florent.xicluna@gmail.com>
parents: 77917
diff changeset
1506 return '{}.{}'.format(base, name) if name else base
74983
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using
Brett Cannon <brett@python.org>
parents: 74935
diff changeset
1507
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using
Brett Cannon <brett@python.org>
parents: 74935
diff changeset
1508
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using
Brett Cannon <brett@python.org>
parents: 74935
diff changeset
1509 def _find_module(name, path):
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using
Brett Cannon <brett@python.org>
parents: 74935
diff changeset
1510 """Find a module's loader."""
76579
3bd60cc27664 Issue #14605: Stop having implicit entries for sys.meta_path.
Brett Cannon <brett@python.org>
parents: 76566
diff changeset
1511 if not sys.meta_path:
3bd60cc27664 Issue #14605: Stop having implicit entries for sys.meta_path.
Brett Cannon <brett@python.org>
parents: 76566
diff changeset
1512 _warnings.warn('sys.meta_path is empty', ImportWarning)
3bd60cc27664 Issue #14605: Stop having implicit entries for sys.meta_path.
Brett Cannon <brett@python.org>
parents: 76566
diff changeset
1513 for finder in sys.meta_path:
77013
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
1514 with _ImportLockContext():
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
1515 loader = finder.find_module(name, path)
74983
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using
Brett Cannon <brett@python.org>
parents: 74935
diff changeset
1516 if loader is not None:
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using
Brett Cannon <brett@python.org>
parents: 74935
diff changeset
1517 # The parent import may have already imported this module.
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using
Brett Cannon <brett@python.org>
parents: 74935
diff changeset
1518 if name not in sys.modules:
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using
Brett Cannon <brett@python.org>
parents: 74935
diff changeset
1519 return loader
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using
Brett Cannon <brett@python.org>
parents: 74935
diff changeset
1520 else:
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using
Brett Cannon <brett@python.org>
parents: 74935
diff changeset
1521 return sys.modules[name].__loader__
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using
Brett Cannon <brett@python.org>
parents: 74935
diff changeset
1522 else:
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using
Brett Cannon <brett@python.org>
parents: 74935
diff changeset
1523 return None
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using
Brett Cannon <brett@python.org>
parents: 74935
diff changeset
1524
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using
Brett Cannon <brett@python.org>
parents: 74935
diff changeset
1525
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using
Brett Cannon <brett@python.org>
parents: 74935
diff changeset
1526 def _sanity_check(name, package, level):
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using
Brett Cannon <brett@python.org>
parents: 74935
diff changeset
1527 """Verify arguments are "sane"."""
75228
909935a236e3 Do a type check instead of an interface check.
Brett Cannon <brett@python.org>
parents: 75193
diff changeset
1528 if not isinstance(name, str):
84582
33d379c2bb79 Issue #18415: Normalize what type of quotes are used with string
Brett Cannon <brett@python.org>
parents: 84468
diff changeset
1529 raise TypeError('module name must be str, not {}'.format(type(name)))
75193
f95faebf5dea Refactor importlib to make it easier to re-implement in C.
Brett Cannon <brett@python.org>
parents: 75185
diff changeset
1530 if level < 0:
f95faebf5dea Refactor importlib to make it easier to re-implement in C.
Brett Cannon <brett@python.org>
parents: 75185
diff changeset
1531 raise ValueError('level must be >= 0')
74983
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using
Brett Cannon <brett@python.org>
parents: 74935
diff changeset
1532 if package:
75228
909935a236e3 Do a type check instead of an interface check.
Brett Cannon <brett@python.org>
parents: 75193
diff changeset
1533 if not isinstance(package, str):
84582
33d379c2bb79 Issue #18415: Normalize what type of quotes are used with string
Brett Cannon <brett@python.org>
parents: 84468
diff changeset
1534 raise TypeError('__package__ not set to a string')
74983
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using
Brett Cannon <brett@python.org>
parents: 74935
diff changeset
1535 elif package not in sys.modules:
84582
33d379c2bb79 Issue #18415: Normalize what type of quotes are used with string
Brett Cannon <brett@python.org>
parents: 84468
diff changeset
1536 msg = ('Parent module {!r} not loaded, cannot perform relative '
33d379c2bb79 Issue #18415: Normalize what type of quotes are used with string
Brett Cannon <brett@python.org>
parents: 84468
diff changeset
1537 'import')
74983
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using
Brett Cannon <brett@python.org>
parents: 74935
diff changeset
1538 raise SystemError(msg.format(package))
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using
Brett Cannon <brett@python.org>
parents: 74935
diff changeset
1539 if not name and level == 0:
84582
33d379c2bb79 Issue #18415: Normalize what type of quotes are used with string
Brett Cannon <brett@python.org>
parents: 84468
diff changeset
1540 raise ValueError('Empty module name')
52141
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now.
Brett Cannon <bcannon@gmail.com>
parents: 52140
diff changeset
1541
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
1542
84464
0d590683c8d1 Issue #18364: Stop using the ImportError._not_found hack.
Brett Cannon <brett@python.org>
parents: 84445
diff changeset
1543 _ERR_MSG_PREFIX = 'No module named '
0d590683c8d1 Issue #18364: Stop using the ImportError._not_found hack.
Brett Cannon <brett@python.org>
parents: 84445
diff changeset
1544 _ERR_MSG = _ERR_MSG_PREFIX + '{!r}'
51639
ef2cba80c975 Change importlib.machinery.PathFinder to not have implicit semantics (that's
Brett Cannon <bcannon@gmail.com>
parents: 51433
diff changeset
1545
77013
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
1546 def _find_and_load_unlocked(name, import_):
74983
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using
Brett Cannon <brett@python.org>
parents: 74935
diff changeset
1547 path = None
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using
Brett Cannon <brett@python.org>
parents: 74935
diff changeset
1548 parent = name.rpartition('.')[0]
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using
Brett Cannon <brett@python.org>
parents: 74935
diff changeset
1549 if parent:
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using
Brett Cannon <brett@python.org>
parents: 74935
diff changeset
1550 if parent not in sys.modules:
78344
62033490ca0f Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise
Nick Coghlan <ncoghlan@gmail.com>
parents: 78314
diff changeset
1551 _call_with_frames_removed(import_, parent)
76081
a40cd5976215 If a module injects something into sys.modules as a side-effect of
Brett Cannon <brett@python.org>
parents: 75463
diff changeset
1552 # Crazy side-effects!
a40cd5976215 If a module injects something into sys.modules as a side-effect of
Brett Cannon <brett@python.org>
parents: 75463
diff changeset
1553 if name in sys.modules:
a40cd5976215 If a module injects something into sys.modules as a side-effect of
Brett Cannon <brett@python.org>
parents: 75463
diff changeset
1554 return sys.modules[name]
74983
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using
Brett Cannon <brett@python.org>
parents: 74935
diff changeset
1555 # Backwards-compatibility; be nicer to skip the dict lookup.
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using
Brett Cannon <brett@python.org>
parents: 74935
diff changeset
1556 parent_module = sys.modules[parent]
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using
Brett Cannon <brett@python.org>
parents: 74935
diff changeset
1557 try:
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using
Brett Cannon <brett@python.org>
parents: 74935
diff changeset
1558 path = parent_module.__path__
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using
Brett Cannon <brett@python.org>
parents: 74935
diff changeset
1559 except AttributeError:
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using
Brett Cannon <brett@python.org>
parents: 74935
diff changeset
1560 msg = (_ERR_MSG + '; {} is not a package').format(name, parent)
76280
c071af95772a Have importlib take advantage of ImportError's new 'name' and 'path'
Brett Cannon <brett@python.org>
parents: 76081
diff changeset
1561 raise ImportError(msg, name=name)
75193
f95faebf5dea Refactor importlib to make it easier to re-implement in C.
Brett Cannon <brett@python.org>
parents: 75185
diff changeset
1562 loader = _find_module(name, path)
f95faebf5dea Refactor importlib to make it easier to re-implement in C.
Brett Cannon <brett@python.org>
parents: 75185
diff changeset
1563 if loader is None:
84464
0d590683c8d1 Issue #18364: Stop using the ImportError._not_found hack.
Brett Cannon <brett@python.org>
parents: 84445
diff changeset
1564 raise ImportError(_ERR_MSG.format(name), name=name)
76390
db5e3431ee4c rollback 005fd1fe31ab (see #14609 and #14582)
Benjamin Peterson <benjamin@python.org>
parents: 76384
diff changeset
1565 elif name not in sys.modules:
db5e3431ee4c rollback 005fd1fe31ab (see #14609 and #14582)
Benjamin Peterson <benjamin@python.org>
parents: 76384
diff changeset
1566 # The parent import may have already imported this module.
db5e3431ee4c rollback 005fd1fe31ab (see #14609 and #14582)
Benjamin Peterson <benjamin@python.org>
parents: 76384
diff changeset
1567 loader.load_module(name)
76753
59b8a7fcb047 Properly mark names in importlib._bootstrap as private.
Brett Cannon <brett@python.org>
parents: 76586
diff changeset
1568 _verbose_message('import {!r} # {!r}', name, loader)
76390
db5e3431ee4c rollback 005fd1fe31ab (see #14609 and #14582)
Benjamin Peterson <benjamin@python.org>
parents: 76384
diff changeset
1569 # Backwards-compatibility; be nicer to skip the dict lookup.
db5e3431ee4c rollback 005fd1fe31ab (see #14609 and #14582)
Benjamin Peterson <benjamin@python.org>
parents: 76384
diff changeset
1570 module = sys.modules[name]
75193
f95faebf5dea Refactor importlib to make it easier to re-implement in C.
Brett Cannon <brett@python.org>
parents: 75185
diff changeset
1571 if parent:
f95faebf5dea Refactor importlib to make it easier to re-implement in C.
Brett Cannon <brett@python.org>
parents: 75185
diff changeset
1572 # Set the module as an attribute on its parent.
f95faebf5dea Refactor importlib to make it easier to re-implement in C.
Brett Cannon <brett@python.org>
parents: 75185
diff changeset
1573 parent_module = sys.modules[parent]
f95faebf5dea Refactor importlib to make it easier to re-implement in C.
Brett Cannon <brett@python.org>
parents: 75185
diff changeset
1574 setattr(parent_module, name.rpartition('.')[2], module)
f95faebf5dea Refactor importlib to make it easier to re-implement in C.
Brett Cannon <brett@python.org>
parents: 75185
diff changeset
1575 # Set __package__ if the loader did not.
77964
37850f083af5 Minor refactoring in importlib._bootstrap, and fix the '_wrap' docstring.
Florent Xicluna <florent.xicluna@gmail.com>
parents: 77917
diff changeset
1576 if getattr(module, '__package__', None) is None:
75193
f95faebf5dea Refactor importlib to make it easier to re-implement in C.
Brett Cannon <brett@python.org>
parents: 75185
diff changeset
1577 try:
f95faebf5dea Refactor importlib to make it easier to re-implement in C.
Brett Cannon <brett@python.org>
parents: 75185
diff changeset
1578 module.__package__ = module.__name__
f95faebf5dea Refactor importlib to make it easier to re-implement in C.
Brett Cannon <brett@python.org>
parents: 75185
diff changeset
1579 if not hasattr(module, '__path__'):
f95faebf5dea Refactor importlib to make it easier to re-implement in C.
Brett Cannon <brett@python.org>
parents: 75185
diff changeset
1580 module.__package__ = module.__package__.rpartition('.')[0]
f95faebf5dea Refactor importlib to make it easier to re-implement in C.
Brett Cannon <brett@python.org>
parents: 75185
diff changeset
1581 except AttributeError:
f95faebf5dea Refactor importlib to make it easier to re-implement in C.
Brett Cannon <brett@python.org>
parents: 75185
diff changeset
1582 pass
76586
496c68f90a03 Issue #14646: __import__() now sets __loader__ if need be.
Brett Cannon <brett@python.org>
parents: 76582
diff changeset
1583 # Set loader if need be.
82643
7647aae9481c Issue #17117: Have both import itself and importlib.util.set_loader()
Brett Cannon <brett@python.org>
parents: 82385
diff changeset
1584 if getattr(module, '__loader__', None) is None:
76586
496c68f90a03 Issue #14646: __import__() now sets __loader__ if need be.
Brett Cannon <brett@python.org>
parents: 76582
diff changeset
1585 try:
496c68f90a03 Issue #14646: __import__() now sets __loader__ if need be.
Brett Cannon <brett@python.org>
parents: 76582
diff changeset
1586 module.__loader__ = loader
496c68f90a03 Issue #14646: __import__() now sets __loader__ if need be.
Brett Cannon <brett@python.org>
parents: 76582
diff changeset
1587 except AttributeError:
496c68f90a03 Issue #14646: __import__() now sets __loader__ if need be.
Brett Cannon <brett@python.org>
parents: 76582
diff changeset
1588 pass
75193
f95faebf5dea Refactor importlib to make it easier to re-implement in C.
Brett Cannon <brett@python.org>
parents: 75185
diff changeset
1589 return module
74983
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using
Brett Cannon <brett@python.org>
parents: 74935
diff changeset
1590
66226
af13fb6fb3df Fix a minor inconsistency in capitalization for the 'No module named' exception
Brett Cannon <bcannon@gmail.com>
parents: 64407
diff changeset
1591
77013
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
1592 def _find_and_load(name, import_):
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
1593 """Find and load the module, and release the import lock."""
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
1594 try:
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
1595 lock = _get_module_lock(name)
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
1596 finally:
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
1597 _imp.release_lock()
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
1598 lock.acquire()
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
1599 try:
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
1600 return _find_and_load_unlocked(name, import_)
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
1601 finally:
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
1602 lock.release()
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
1603
66226
af13fb6fb3df Fix a minor inconsistency in capitalization for the 'No module named' exception
Brett Cannon <bcannon@gmail.com>
parents: 64407
diff changeset
1604
51383
a00e5dcea16c Initial, untested stab at writing a common denominator function for __import__
Brett Cannon <bcannon@gmail.com>
parents: 51356
diff changeset
1605 def _gcd_import(name, package=None, level=0):
a00e5dcea16c Initial, untested stab at writing a common denominator function for __import__
Brett Cannon <bcannon@gmail.com>
parents: 51356
diff changeset
1606 """Import and return the module based on its name, the package the call is
a00e5dcea16c Initial, untested stab at writing a common denominator function for __import__
Brett Cannon <bcannon@gmail.com>
parents: 51356
diff changeset
1607 being made from, and the level adjustment.
a00e5dcea16c Initial, untested stab at writing a common denominator function for __import__
Brett Cannon <bcannon@gmail.com>
parents: 51356
diff changeset
1608
a00e5dcea16c Initial, untested stab at writing a common denominator function for __import__
Brett Cannon <bcannon@gmail.com>
parents: 51356
diff changeset
1609 This function represents the greatest common denominator of functionality
68447
Eric V. Smith <eric@trueblade.com>
parents: 68257
diff changeset
1610 between import_module and __import__. This includes setting __package__ if
51426
7a1356d2aeef Rewrite the code implementing __import__ for importlib. Now it is much simpler
Brett Cannon <bcannon@gmail.com>
parents: 51383
diff changeset
1611 the loader did not.
7a1356d2aeef Rewrite the code implementing __import__ for importlib. Now it is much simpler
Brett Cannon <bcannon@gmail.com>
parents: 51383
diff changeset
1612
51383
a00e5dcea16c Initial, untested stab at writing a common denominator function for __import__
Brett Cannon <bcannon@gmail.com>
parents: 51356
diff changeset
1613 """
74983
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using
Brett Cannon <brett@python.org>
parents: 74935
diff changeset
1614 _sanity_check(name, package, level)
51383
a00e5dcea16c Initial, untested stab at writing a common denominator function for __import__
Brett Cannon <bcannon@gmail.com>
parents: 51356
diff changeset
1615 if level > 0:
74983
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using
Brett Cannon <brett@python.org>
parents: 74935
diff changeset
1616 name = _resolve_name(name, package, level)
77013
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
1617 _imp.acquire_lock()
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
1618 if name not in sys.modules:
75193
f95faebf5dea Refactor importlib to make it easier to re-implement in C.
Brett Cannon <brett@python.org>
parents: 75185
diff changeset
1619 return _find_and_load(name, _gcd_import)
77013
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
1620 module = sys.modules[name]
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
1621 if module is None:
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
1622 _imp.release_lock()
84582
33d379c2bb79 Issue #18415: Normalize what type of quotes are used with string
Brett Cannon <brett@python.org>
parents: 84468
diff changeset
1623 message = ('import of {} halted; '
33d379c2bb79 Issue #18415: Normalize what type of quotes are used with string
Brett Cannon <brett@python.org>
parents: 84468
diff changeset
1624 'None in sys.modules'.format(name))
77013
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
1625 raise ImportError(message, name=name)
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
1626 _lock_unlock_module(name)
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
1627 return module
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
1628
75235
3670df23081d Turn _return_module() into _handle_fromlist().
Brett Cannon <brett@python.org>
parents: 75229
diff changeset
1629 def _handle_fromlist(module, fromlist, import_):
74983
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using
Brett Cannon <brett@python.org>
parents: 74935
diff changeset
1630 """Figure out what __import__ should return.
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
1631
74983
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using
Brett Cannon <brett@python.org>
parents: 74935
diff changeset
1632 The import_ parameter is a callable which takes the name of module to
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using
Brett Cannon <brett@python.org>
parents: 74935
diff changeset
1633 import. It is required to decouple the function from assuming importlib's
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using
Brett Cannon <brett@python.org>
parents: 74935
diff changeset
1634 import implementation is desired.
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
1635
51426
7a1356d2aeef Rewrite the code implementing __import__ for importlib. Now it is much simpler
Brett Cannon <bcannon@gmail.com>
parents: 51383
diff changeset
1636 """
7a1356d2aeef Rewrite the code implementing __import__ for importlib. Now it is much simpler
Brett Cannon <bcannon@gmail.com>
parents: 51383
diff changeset
1637 # The hell that is fromlist ...
75235
3670df23081d Turn _return_module() into _handle_fromlist().
Brett Cannon <brett@python.org>
parents: 75229
diff changeset
1638 # If a package was imported, try to import stuff from fromlist.
3670df23081d Turn _return_module() into _handle_fromlist().
Brett Cannon <brett@python.org>
parents: 75229
diff changeset
1639 if hasattr(module, '__path__'):
78051
dc18a2a66d16 Issue #15111: When a module was imported using a 'from import'
Brett Cannon <brett@python.org>
parents: 78042
diff changeset
1640 if '*' in fromlist:
75235
3670df23081d Turn _return_module() into _handle_fromlist().
Brett Cannon <brett@python.org>
parents: 75229
diff changeset
1641 fromlist = list(fromlist)
3670df23081d Turn _return_module() into _handle_fromlist().
Brett Cannon <brett@python.org>
parents: 75229
diff changeset
1642 fromlist.remove('*')
78051
dc18a2a66d16 Issue #15111: When a module was imported using a 'from import'
Brett Cannon <brett@python.org>
parents: 78042
diff changeset
1643 if hasattr(module, '__all__'):
dc18a2a66d16 Issue #15111: When a module was imported using a 'from import'
Brett Cannon <brett@python.org>
parents: 78042
diff changeset
1644 fromlist.extend(module.__all__)
77964
37850f083af5 Minor refactoring in importlib._bootstrap, and fix the '_wrap' docstring.
Florent Xicluna <florent.xicluna@gmail.com>
parents: 77917
diff changeset
1645 for x in fromlist:
37850f083af5 Minor refactoring in importlib._bootstrap, and fix the '_wrap' docstring.
Florent Xicluna <florent.xicluna@gmail.com>
parents: 77917
diff changeset
1646 if not hasattr(module, x):
79652
09b5158d5284 Closes issue #15111: Calling __import__ with a module specified in
Brett Cannon <brett@python.org>
parents: 79228
diff changeset
1647 from_name = '{}.{}'.format(module.__name__, x)
78619
0d52f125dd32 Issue #15715: Ignore failed imports triggered by the use of fromlist.
Brett Cannon <brett@python.org>
parents: 78511
diff changeset
1648 try:
79652
09b5158d5284 Closes issue #15111: Calling __import__ with a module specified in
Brett Cannon <brett@python.org>
parents: 79228
diff changeset
1649 _call_with_frames_removed(import_, from_name)
78758
ca4bf8e10bc0 Issue #15316: Let exceptions raised during imports triggered by the
Brett Cannon <brett@python.org>
parents: 78738
diff changeset
1650 except ImportError as exc:
78619
0d52f125dd32 Issue #15715: Ignore failed imports triggered by the use of fromlist.
Brett Cannon <brett@python.org>
parents: 78511
diff changeset
1651 # Backwards-compatibility dictates we ignore failed
78758
ca4bf8e10bc0 Issue #15316: Let exceptions raised during imports triggered by the
Brett Cannon <brett@python.org>
parents: 78738
diff changeset
1652 # imports triggered by fromlist for modules that don't
ca4bf8e10bc0 Issue #15316: Let exceptions raised during imports triggered by the
Brett Cannon <brett@python.org>
parents: 78738
diff changeset
1653 # exist.
84464
0d590683c8d1 Issue #18364: Stop using the ImportError._not_found hack.
Brett Cannon <brett@python.org>
parents: 84445
diff changeset
1654 if str(exc).startswith(_ERR_MSG_PREFIX):
79652
09b5158d5284 Closes issue #15111: Calling __import__ with a module specified in
Brett Cannon <brett@python.org>
parents: 79228
diff changeset
1655 if exc.name == from_name:
09b5158d5284 Closes issue #15111: Calling __import__ with a module specified in
Brett Cannon <brett@python.org>
parents: 79228
diff changeset
1656 continue
09b5158d5284 Closes issue #15111: Calling __import__ with a module specified in
Brett Cannon <brett@python.org>
parents: 79228
diff changeset
1657 raise
75235
3670df23081d Turn _return_module() into _handle_fromlist().
Brett Cannon <brett@python.org>
parents: 75229
diff changeset
1658 return module
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
1659
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
1660
74983
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using
Brett Cannon <brett@python.org>
parents: 74935
diff changeset
1661 def _calc___package__(globals):
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using
Brett Cannon <brett@python.org>
parents: 74935
diff changeset
1662 """Calculate what __package__ should be.
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using
Brett Cannon <brett@python.org>
parents: 74935
diff changeset
1663
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using
Brett Cannon <brett@python.org>
parents: 74935
diff changeset
1664 __package__ is not guaranteed to be defined or could be set to None
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using
Brett Cannon <brett@python.org>
parents: 74935
diff changeset
1665 to represent that its proper value is unknown.
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using
Brett Cannon <brett@python.org>
parents: 74935
diff changeset
1666
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using
Brett Cannon <brett@python.org>
parents: 74935
diff changeset
1667 """
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using
Brett Cannon <brett@python.org>
parents: 74935
diff changeset
1668 package = globals.get('__package__')
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using
Brett Cannon <brett@python.org>
parents: 74935
diff changeset
1669 if package is None:
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using
Brett Cannon <brett@python.org>
parents: 74935
diff changeset
1670 package = globals['__name__']
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using
Brett Cannon <brett@python.org>
parents: 74935
diff changeset
1671 if '__path__' not in globals:
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using
Brett Cannon <brett@python.org>
parents: 74935
diff changeset
1672 package = package.rpartition('.')[0]
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using
Brett Cannon <brett@python.org>
parents: 74935
diff changeset
1673 return package
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using
Brett Cannon <brett@python.org>
parents: 74935
diff changeset
1674
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using
Brett Cannon <brett@python.org>
parents: 74935
diff changeset
1675
78308
bd58c421057c Issue #14578: Support modules registered in the Windows registry again.
Martin v. Löwis <martin@v.loewis.de>
parents: 78112
diff changeset
1676 def _get_supported_file_loaders():
bd58c421057c Issue #14578: Support modules registered in the Windows registry again.
Martin v. Löwis <martin@v.loewis.de>
parents: 78112
diff changeset
1677 """Returns a list of file-based module loaders.
bd58c421057c Issue #14578: Support modules registered in the Windows registry again.
Martin v. Löwis <martin@v.loewis.de>
parents: 78112
diff changeset
1678
85945
32b18998a560 [issue19151] Fix docstring and use of _get_suppported_file_loaders() to reflect 2-tuples.
Eric Snow <ericsnowcurrently@gmail.com>
parents: 85518
diff changeset
1679 Each item is a tuple (loader, suffixes).
78308
bd58c421057c Issue #14578: Support modules registered in the Windows registry again.
Martin v. Löwis <martin@v.loewis.de>
parents: 78112
diff changeset
1680 """
78486
1db6553f3f8c Issue #15576: Allow extension modules to be a package's __init__
Brett Cannon <brett@python.org>
parents: 78485
diff changeset
1681 extensions = ExtensionFileLoader, _imp.extension_suffixes()
1db6553f3f8c Issue #15576: Allow extension modules to be a package's __init__
Brett Cannon <brett@python.org>
parents: 78485
diff changeset
1682 source = SourceFileLoader, SOURCE_SUFFIXES
1db6553f3f8c Issue #15576: Allow extension modules to be a package's __init__
Brett Cannon <brett@python.org>
parents: 78485
diff changeset
1683 bytecode = SourcelessFileLoader, BYTECODE_SUFFIXES
78308
bd58c421057c Issue #14578: Support modules registered in the Windows registry again.
Martin v. Löwis <martin@v.loewis.de>
parents: 78112
diff changeset
1684 return [extensions, source, bytecode]
bd58c421057c Issue #14578: Support modules registered in the Windows registry again.
Martin v. Löwis <martin@v.loewis.de>
parents: 78112
diff changeset
1685
bd58c421057c Issue #14578: Support modules registered in the Windows registry again.
Martin v. Löwis <martin@v.loewis.de>
parents: 78112
diff changeset
1686
78456
4240282a9f4a Issue #15471: Don't use mutable object as default values for the
Brett Cannon <brett@python.org>
parents: 78382
diff changeset
1687 def __import__(name, globals=None, locals=None, fromlist=(), level=0):
51426
7a1356d2aeef Rewrite the code implementing __import__ for importlib. Now it is much simpler
Brett Cannon <bcannon@gmail.com>
parents: 51383
diff changeset
1688 """Import a module.
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
1689
51426
7a1356d2aeef Rewrite the code implementing __import__ for importlib. Now it is much simpler
Brett Cannon <bcannon@gmail.com>
parents: 51383
diff changeset
1690 The 'globals' argument is used to infer where the import is occuring from
7a1356d2aeef Rewrite the code implementing __import__ for importlib. Now it is much simpler
Brett Cannon <bcannon@gmail.com>
parents: 51383
diff changeset
1691 to handle relative imports. The 'locals' argument is ignored. The
7a1356d2aeef Rewrite the code implementing __import__ for importlib. Now it is much simpler
Brett Cannon <bcannon@gmail.com>
parents: 51383
diff changeset
1692 'fromlist' argument specifies what should exist as attributes on the module
7a1356d2aeef Rewrite the code implementing __import__ for importlib. Now it is much simpler
Brett Cannon <bcannon@gmail.com>
parents: 51383
diff changeset
1693 being imported (e.g. ``from module import <fromlist>``). The 'level'
7a1356d2aeef Rewrite the code implementing __import__ for importlib. Now it is much simpler
Brett Cannon <bcannon@gmail.com>
parents: 51383
diff changeset
1694 argument represents the package location to import from in a relative
7a1356d2aeef Rewrite the code implementing __import__ for importlib. Now it is much simpler
Brett Cannon <bcannon@gmail.com>
parents: 51383
diff changeset
1695 import (e.g. ``from ..pkg import mod`` would have a 'level' of 2).
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
1696
51426
7a1356d2aeef Rewrite the code implementing __import__ for importlib. Now it is much simpler
Brett Cannon <bcannon@gmail.com>
parents: 51383
diff changeset
1697 """
7a1356d2aeef Rewrite the code implementing __import__ for importlib. Now it is much simpler
Brett Cannon <bcannon@gmail.com>
parents: 51383
diff changeset
1698 if level == 0:
7a1356d2aeef Rewrite the code implementing __import__ for importlib. Now it is much simpler
Brett Cannon <bcannon@gmail.com>
parents: 51383
diff changeset
1699 module = _gcd_import(name)
7a1356d2aeef Rewrite the code implementing __import__ for importlib. Now it is much simpler
Brett Cannon <bcannon@gmail.com>
parents: 51383
diff changeset
1700 else:
78456
4240282a9f4a Issue #15471: Don't use mutable object as default values for the
Brett Cannon <brett@python.org>
parents: 78382
diff changeset
1701 globals_ = globals if globals is not None else {}
4240282a9f4a Issue #15471: Don't use mutable object as default values for the
Brett Cannon <brett@python.org>
parents: 78382
diff changeset
1702 package = _calc___package__(globals_)
51426
7a1356d2aeef Rewrite the code implementing __import__ for importlib. Now it is much simpler
Brett Cannon <bcannon@gmail.com>
parents: 51383
diff changeset
1703 module = _gcd_import(name, package, level)
7a1356d2aeef Rewrite the code implementing __import__ for importlib. Now it is much simpler
Brett Cannon <bcannon@gmail.com>
parents: 51383
diff changeset
1704 if not fromlist:
7a1356d2aeef Rewrite the code implementing __import__ for importlib. Now it is much simpler
Brett Cannon <bcannon@gmail.com>
parents: 51383
diff changeset
1705 # Return up to the first dot in 'name'. This is complicated by the fact
7a1356d2aeef Rewrite the code implementing __import__ for importlib. Now it is much simpler
Brett Cannon <bcannon@gmail.com>
parents: 51383
diff changeset
1706 # that 'name' may be relative.
7a1356d2aeef Rewrite the code implementing __import__ for importlib. Now it is much simpler
Brett Cannon <bcannon@gmail.com>
parents: 51383
diff changeset
1707 if level == 0:
76826
d6324941b739 Issue #14583: Fix importlib bug when a package's __init__.py would first import one of its modules then raise an error.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76765
diff changeset
1708 return _gcd_import(name.partition('.')[0])
51426
7a1356d2aeef Rewrite the code implementing __import__ for importlib. Now it is much simpler
Brett Cannon <bcannon@gmail.com>
parents: 51383
diff changeset
1709 elif not name:
7a1356d2aeef Rewrite the code implementing __import__ for importlib. Now it is much simpler
Brett Cannon <bcannon@gmail.com>
parents: 51383
diff changeset
1710 return module
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
1711 else:
79228
4a979326e6d6 Add some comments.
Brett Cannon <brett@python.org>
parents: 79191
diff changeset
1712 # Figure out where to slice the module's name up to the first dot
4a979326e6d6 Add some comments.
Brett Cannon <brett@python.org>
parents: 79191
diff changeset
1713 # in 'name'.
51426
7a1356d2aeef Rewrite the code implementing __import__ for importlib. Now it is much simpler
Brett Cannon <bcannon@gmail.com>
parents: 51383
diff changeset
1714 cut_off = len(name) - len(name.partition('.')[0])
79228
4a979326e6d6 Add some comments.
Brett Cannon <brett@python.org>
parents: 79191
diff changeset
1715 # Slice end needs to be positive to alleviate need to special-case
4a979326e6d6 Add some comments.
Brett Cannon <brett@python.org>
parents: 79191
diff changeset
1716 # when ``'.' not in name``.
76312
9e8cbf07068a Handle importing pkg.mod by executing
Brett Cannon <brett@python.org>
parents: 76309
diff changeset
1717 return sys.modules[module.__name__[:len(module.__name__)-cut_off]]
75235
3670df23081d Turn _return_module() into _handle_fromlist().
Brett Cannon <brett@python.org>
parents: 75229
diff changeset
1718 else:
3670df23081d Turn _return_module() into _handle_fromlist().
Brett Cannon <brett@python.org>
parents: 75229
diff changeset
1719 return _handle_fromlist(module, fromlist, _gcd_import)
74983
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using
Brett Cannon <brett@python.org>
parents: 74935
diff changeset
1720
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using
Brett Cannon <brett@python.org>
parents: 74935
diff changeset
1721
76445
c820aa9c0c00 Issue #13959: Keep imp.get_magic() in C code, but cache in importlib
Brett Cannon <brett@python.org>
parents: 76428
diff changeset
1722
76331
d777f854a66e Issue #13959: Rename imp to _imp and add Lib/imp.py and begin
Brett Cannon <brett@python.org>
parents: 76330
diff changeset
1723 def _setup(sys_module, _imp_module):
74839
322e6f818cd2 Move setup code from importlib.__init__ to
Brett Cannon <brett@python.org>
parents: 74838
diff changeset
1724 """Setup importlib by importing needed built-in modules and injecting them
322e6f818cd2 Move setup code from importlib.__init__ to
Brett Cannon <brett@python.org>
parents: 74838
diff changeset
1725 into the global namespace.
322e6f818cd2 Move setup code from importlib.__init__ to
Brett Cannon <brett@python.org>
parents: 74838
diff changeset
1726
76331
d777f854a66e Issue #13959: Rename imp to _imp and add Lib/imp.py and begin
Brett Cannon <brett@python.org>
parents: 76330
diff changeset
1727 As sys is needed for sys.modules access and _imp is needed to load built-in
74935
81f3bb5efa40 Bring importlib in line w/ changes made in my personal bootstrap branch in the sandbox.
Brett Cannon <brett@python.org>
parents: 74867
diff changeset
1728 modules, those two modules must be explicitly passed in.
74839
322e6f818cd2 Move setup code from importlib.__init__ to
Brett Cannon <brett@python.org>
parents: 74838
diff changeset
1729
322e6f818cd2 Move setup code from importlib.__init__ to
Brett Cannon <brett@python.org>
parents: 74838
diff changeset
1730 """
79173
4de5e4ec3cff don't depend on __debug__ because it's baked in at freeze time (issue #16046)
Benjamin Peterson <benjamin@python.org>
parents: 78779
diff changeset
1731 global _imp, sys, BYTECODE_SUFFIXES
76331
d777f854a66e Issue #13959: Rename imp to _imp and add Lib/imp.py and begin
Brett Cannon <brett@python.org>
parents: 76330
diff changeset
1732 _imp = _imp_module
74839
322e6f818cd2 Move setup code from importlib.__init__ to
Brett Cannon <brett@python.org>
parents: 74838
diff changeset
1733 sys = sys_module
322e6f818cd2 Move setup code from importlib.__init__ to
Brett Cannon <brett@python.org>
parents: 74838
diff changeset
1734
79173
4de5e4ec3cff don't depend on __debug__ because it's baked in at freeze time (issue #16046)
Benjamin Peterson <benjamin@python.org>
parents: 78779
diff changeset
1735 if sys.flags.optimize:
4de5e4ec3cff don't depend on __debug__ because it's baked in at freeze time (issue #16046)
Benjamin Peterson <benjamin@python.org>
parents: 78779
diff changeset
1736 BYTECODE_SUFFIXES = OPTIMIZED_BYTECODE_SUFFIXES
51426
7a1356d2aeef Rewrite the code implementing __import__ for importlib. Now it is much simpler
Brett Cannon <bcannon@gmail.com>
parents: 51383
diff changeset
1737 else:
79173
4de5e4ec3cff don't depend on __debug__ because it's baked in at freeze time (issue #16046)
Benjamin Peterson <benjamin@python.org>
parents: 78779
diff changeset
1738 BYTECODE_SUFFIXES = DEBUG_BYTECODE_SUFFIXES
4de5e4ec3cff don't depend on __debug__ because it's baked in at freeze time (issue #16046)
Benjamin Peterson <benjamin@python.org>
parents: 78779
diff changeset
1739
81869
05747d3bcd9c Issue #17098: Make sure every module has __loader__ defined.
Brett Cannon <brett@python.org>
parents: 81427
diff changeset
1740 module_type = type(sys)
81884
19ea454ccdf7 Issue #17098: Be more stringent of setting __loader__ on early imported
Brett Cannon <brett@python.org>
parents: 81869
diff changeset
1741 for name, module in sys.modules.items():
81869
05747d3bcd9c Issue #17098: Make sure every module has __loader__ defined.
Brett Cannon <brett@python.org>
parents: 81427
diff changeset
1742 if isinstance(module, module_type):
83607
e39a8f8ceb9f #17115,17116: Have modules initialize the __package__ and __loader__
Brett Cannon <brett@python.org>
parents: 83572
diff changeset
1743 if getattr(module, '__loader__', None) is None:
81884
19ea454ccdf7 Issue #17098: Be more stringent of setting __loader__ on early imported
Brett Cannon <brett@python.org>
parents: 81869
diff changeset
1744 if name in sys.builtin_module_names:
19ea454ccdf7 Issue #17098: Be more stringent of setting __loader__ on early imported
Brett Cannon <brett@python.org>
parents: 81869
diff changeset
1745 module.__loader__ = BuiltinImporter
19ea454ccdf7 Issue #17098: Be more stringent of setting __loader__ on early imported
Brett Cannon <brett@python.org>
parents: 81869
diff changeset
1746 elif _imp.is_frozen(name):
19ea454ccdf7 Issue #17098: Be more stringent of setting __loader__ on early imported
Brett Cannon <brett@python.org>
parents: 81869
diff changeset
1747 module.__loader__ = FrozenImporter
74839
322e6f818cd2 Move setup code from importlib.__init__ to
Brett Cannon <brett@python.org>
parents: 74838
diff changeset
1748
322e6f818cd2 Move setup code from importlib.__init__ to
Brett Cannon <brett@python.org>
parents: 74838
diff changeset
1749 self_module = sys.modules[__name__]
322e6f818cd2 Move setup code from importlib.__init__ to
Brett Cannon <brett@python.org>
parents: 74838
diff changeset
1750 for builtin_name in ('_io', '_warnings', 'builtins', 'marshal'):
322e6f818cd2 Move setup code from importlib.__init__ to
Brett Cannon <brett@python.org>
parents: 74838
diff changeset
1751 if builtin_name not in sys.modules:
322e6f818cd2 Move setup code from importlib.__init__ to
Brett Cannon <brett@python.org>
parents: 74838
diff changeset
1752 builtin_module = BuiltinImporter.load_module(builtin_name)
322e6f818cd2 Move setup code from importlib.__init__ to
Brett Cannon <brett@python.org>
parents: 74838
diff changeset
1753 else:
322e6f818cd2 Move setup code from importlib.__init__ to
Brett Cannon <brett@python.org>
parents: 74838
diff changeset
1754 builtin_module = sys.modules[builtin_name]
322e6f818cd2 Move setup code from importlib.__init__ to
Brett Cannon <brett@python.org>
parents: 74838
diff changeset
1755 setattr(self_module, builtin_name, builtin_module)
322e6f818cd2 Move setup code from importlib.__init__ to
Brett Cannon <brett@python.org>
parents: 74838
diff changeset
1756
79480
15949f34e08f #16135: Removal of OS/2 support (Python code partial cleanup)
Jesus Cea <jcea@jcea.es>
parents: 79228
diff changeset
1757 os_details = ('posix', ['/']), ('nt', ['\\', '/'])
76449
163360370ad2 Make path manipulation more robust for platforms with alternative path
Brett Cannon <brett@python.org>
parents: 76445
diff changeset
1758 for builtin_os, path_separators in os_details:
76472
4e853913054c Issue #13959: Continue to try to accomodate altsep in importlib by not
Brett Cannon <brett@python.org>
parents: 76466
diff changeset
1759 # Assumption made in _path_join()
4e853913054c Issue #13959: Continue to try to accomodate altsep in importlib by not
Brett Cannon <brett@python.org>
parents: 76466
diff changeset
1760 assert all(len(sep) == 1 for sep in path_separators)
76449
163360370ad2 Make path manipulation more robust for platforms with alternative path
Brett Cannon <brett@python.org>
parents: 76445
diff changeset
1761 path_sep = path_separators[0]
74839
322e6f818cd2 Move setup code from importlib.__init__ to
Brett Cannon <brett@python.org>
parents: 74838
diff changeset
1762 if builtin_os in sys.modules:
322e6f818cd2 Move setup code from importlib.__init__ to
Brett Cannon <brett@python.org>
parents: 74838
diff changeset
1763 os_module = sys.modules[builtin_os]
322e6f818cd2 Move setup code from importlib.__init__ to
Brett Cannon <brett@python.org>
parents: 74838
diff changeset
1764 break
322e6f818cd2 Move setup code from importlib.__init__ to
Brett Cannon <brett@python.org>
parents: 74838
diff changeset
1765 else:
322e6f818cd2 Move setup code from importlib.__init__ to
Brett Cannon <brett@python.org>
parents: 74838
diff changeset
1766 try:
322e6f818cd2 Move setup code from importlib.__init__ to
Brett Cannon <brett@python.org>
parents: 74838
diff changeset
1767 os_module = BuiltinImporter.load_module(builtin_os)
322e6f818cd2 Move setup code from importlib.__init__ to
Brett Cannon <brett@python.org>
parents: 74838
diff changeset
1768 break
322e6f818cd2 Move setup code from importlib.__init__ to
Brett Cannon <brett@python.org>
parents: 74838
diff changeset
1769 except ImportError:
322e6f818cd2 Move setup code from importlib.__init__ to
Brett Cannon <brett@python.org>
parents: 74838
diff changeset
1770 continue
322e6f818cd2 Move setup code from importlib.__init__ to
Brett Cannon <brett@python.org>
parents: 74838
diff changeset
1771 else:
322e6f818cd2 Move setup code from importlib.__init__ to
Brett Cannon <brett@python.org>
parents: 74838
diff changeset
1772 raise ImportError('importlib requires posix or nt')
77013
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
1773
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
1774 try:
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
1775 thread_module = BuiltinImporter.load_module('_thread')
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
1776 except ImportError:
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
1777 # Python was built without threads
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
1778 thread_module = None
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
1779 weakref_module = BuiltinImporter.load_module('_weakref')
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
1780
78308
bd58c421057c Issue #14578: Support modules registered in the Windows registry again.
Martin v. Löwis <martin@v.loewis.de>
parents: 78112
diff changeset
1781 if builtin_os == 'nt':
bd58c421057c Issue #14578: Support modules registered in the Windows registry again.
Martin v. Löwis <martin@v.loewis.de>
parents: 78112
diff changeset
1782 winreg_module = BuiltinImporter.load_module('winreg')
bd58c421057c Issue #14578: Support modules registered in the Windows registry again.
Martin v. Löwis <martin@v.loewis.de>
parents: 78112
diff changeset
1783 setattr(self_module, '_winreg', winreg_module)
bd58c421057c Issue #14578: Support modules registered in the Windows registry again.
Martin v. Löwis <martin@v.loewis.de>
parents: 78112
diff changeset
1784
74839
322e6f818cd2 Move setup code from importlib.__init__ to
Brett Cannon <brett@python.org>
parents: 74838
diff changeset
1785 setattr(self_module, '_os', os_module)
77013
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
1786 setattr(self_module, '_thread', thread_module)
edb9ce3a6c2e Issue #9260: A finer-grained import lock.
Antoine Pitrou <solipsis@pitrou.net>
parents: 76877
diff changeset
1787 setattr(self_module, '_weakref', weakref_module)
74839
322e6f818cd2 Move setup code from importlib.__init__ to
Brett Cannon <brett@python.org>
parents: 74838
diff changeset
1788 setattr(self_module, 'path_sep', path_sep)
82382
2528e4aea338 Issue #17220: Little cleanup of _bootstrap.py.
Serhiy Storchaka <storchaka@gmail.com>
parents: 82237
diff changeset
1789 setattr(self_module, 'path_separators', ''.join(path_separators))
75048
bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function.
Antoine Pitrou <solipsis@pitrou.net>
parents: 75045
diff changeset
1790 # Constants
75055
b8576719fe71 _relax_case -> _make_relax_case
Antoine Pitrou <solipsis@pitrou.net>
parents: 75048
diff changeset
1791 setattr(self_module, '_relax_case', _make_relax_case())
78486
1db6553f3f8c Issue #15576: Allow extension modules to be a package's __init__
Brett Cannon <brett@python.org>
parents: 78485
diff changeset
1792 EXTENSION_SUFFIXES.extend(_imp.extension_suffixes())
76451
ea46ebba8a0f Issue #13959: Re-implement imp.source_from_cache() in Lib/imp.py.
Brett Cannon <brett@python.org>
parents: 76450
diff changeset
1793 if builtin_os == 'nt':
76874
b81ddaf0db47 Issue #13959: Deprecate imp.get_suffixes() for new attributes on
Brett Cannon <brett@python.org>
parents: 76826
diff changeset
1794 SOURCE_SUFFIXES.append('.pyw')
78486
1db6553f3f8c Issue #15576: Allow extension modules to be a package's __init__
Brett Cannon <brett@python.org>
parents: 78485
diff changeset
1795 if '_d.pyd' in EXTENSION_SUFFIXES:
78380
a1ac1e13c5a0 Close #15519: Properly expose WindowsRegistryFinder in importlib and bring the name into line with normal import terminology. Original patch by Eric Snow
Nick Coghlan <ncoghlan@gmail.com>
parents: 78344
diff changeset
1796 WindowsRegistryFinder.DEBUG_BUILD = True
75000
e477832212b9 Optimize importlib's case-sensitivity check by wasting as little time as possible under case-sensitive OSs.
Brett Cannon <brett@python.org>
parents: 74998
diff changeset
1797
74935
81f3bb5efa40 Bring importlib in line w/ changes made in my personal bootstrap branch in the sandbox.
Brett Cannon <brett@python.org>
parents: 74867
diff changeset
1798
76331
d777f854a66e Issue #13959: Rename imp to _imp and add Lib/imp.py and begin
Brett Cannon <brett@python.org>
parents: 76330
diff changeset
1799 def _install(sys_module, _imp_module):
76765
146001e3364d Clean up a docstring.
Brett Cannon <brett@python.org>
parents: 76755
diff changeset
1800 """Install importlib as the implementation of import."""
76331
d777f854a66e Issue #13959: Rename imp to _imp and add Lib/imp.py and begin
Brett Cannon <brett@python.org>
parents: 76330
diff changeset
1801 _setup(sys_module, _imp_module)
78308
bd58c421057c Issue #14578: Support modules registered in the Windows registry again.
Martin v. Löwis <martin@v.loewis.de>
parents: 78112
diff changeset
1802 supported_loaders = _get_supported_file_loaders()
76582
7025ee00dbf6 Issue #14605: Use None in sys.path_importer_cache to represent no
Brett Cannon <brett@python.org>
parents: 76579
diff changeset
1803 sys.path_hooks.extend([FileFinder.path_hook(*supported_loaders)])
78308
bd58c421057c Issue #14578: Support modules registered in the Windows registry again.
Martin v. Löwis <martin@v.loewis.de>
parents: 78112
diff changeset
1804 sys.meta_path.append(BuiltinImporter)
bd58c421057c Issue #14578: Support modules registered in the Windows registry again.
Martin v. Löwis <martin@v.loewis.de>
parents: 78112
diff changeset
1805 sys.meta_path.append(FrozenImporter)
bd58c421057c Issue #14578: Support modules registered in the Windows registry again.
Martin v. Löwis <martin@v.loewis.de>
parents: 78112
diff changeset
1806 if _os.__name__ == 'nt':
78380
a1ac1e13c5a0 Close #15519: Properly expose WindowsRegistryFinder in importlib and bring the name into line with normal import terminology. Original patch by Eric Snow
Nick Coghlan <ncoghlan@gmail.com>
parents: 78344
diff changeset
1807 sys.meta_path.append(WindowsRegistryFinder)
78308
bd58c421057c Issue #14578: Support modules registered in the Windows registry again.
Martin v. Löwis <martin@v.loewis.de>
parents: 78112
diff changeset
1808 sys.meta_path.append(PathFinder)