cpython-lru_cache-weakref: Lib/importlib/

annotate Lib/importlib/_bootstrap.py @ 84172:e353f64dfd95

importlib.abc.SourceLoader.get_source() was re-raising SyntaxError and UnicodeDecodeError as ImportError. That was over-reaching the point of raising ImportError in get_source() (which is to signal the source code was not found when it should have). Conflating the two exceptions with ImportError could lead to masking errors with the source which should be known outside of whether there was an error simply getting the source to begin with.

author	Brett Cannon <brett@python.org>
date	Sun, 16 Jun 2013 18:05:54 -0400
parents	205aa49e5cd5
children	bdd60bedf933

rev	line source
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	1 """Core implementation of import.
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	2
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	3 This module is NOT meant to be directly imported! It has been designed such
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	4 that it can be bootstrapped into Python as the implementation of import. As
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	5 such it requires the injection of specific modules and attributes in order to
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	6 work. One should use importlib as the public-facing version of this module.
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	7
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	8 """
76549 acfdf46b8de1 Issue #14605 and #14642: Marc-Andre Lemburg <mal@egenix.com> parents: 76548 diff changeset	9 #
acfdf46b8de1 Issue #14605 and #14642: Marc-Andre Lemburg <mal@egenix.com> parents: 76548 diff changeset	10 # IMPORTANT: Whenever making changes to this module, be sure to run
acfdf46b8de1 Issue #14605 and #14642: Marc-Andre Lemburg <mal@egenix.com> parents: 76548 diff changeset	11 # a top-level make in order to get the frozen version of the module
83960 3a1976680717 Introduce importlib.util.ModuleManager which is a context manager to Brett Cannon <brett@python.org> parents: 83791 diff changeset	12 # update. Not doing so will result in the Makefile to fail for
76549 acfdf46b8de1 Issue #14605 and #14642: Marc-Andre Lemburg <mal@egenix.com> parents: 76548 diff changeset	13 # all others who don't have a ./python around to freeze the module
acfdf46b8de1 Issue #14605 and #14642: Marc-Andre Lemburg <mal@egenix.com> parents: 76548 diff changeset	14 # in the early stages of compilation.
acfdf46b8de1 Issue #14605 and #14642: Marc-Andre Lemburg <mal@egenix.com> parents: 76548 diff changeset	15 #
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	16
76449 163360370ad2 Make path manipulation more robust for platforms with alternative path Brett Cannon <brett@python.org> parents: 76445 diff changeset	17 # See importlib._setup() for what is injected into the global namespace.
163360370ad2 Make path manipulation more robust for platforms with alternative path Brett Cannon <brett@python.org> parents: 76445 diff changeset	18
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	19 # When editing this code be aware that code executed at import time CANNOT
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	20 # reference any injected objects! This includes not only global code but also
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	21 # anything specified at the class level.
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	22
52140 1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap. Brett Cannon <bcannon@gmail.com> parents: 52139 diff changeset	23 # Bootstrap-related code ######################################################
1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap. Brett Cannon <bcannon@gmail.com> parents: 52139 diff changeset	24
76753 59b8a7fcb047 Properly mark names in importlib._bootstrap as private. Brett Cannon <brett@python.org> parents: 76586 diff changeset	25 _CASE_INSENSITIVE_PLATFORMS = 'win', 'cygwin', 'darwin'
74643 ecf4a7bb8807 Relocate importlib._case_ok to importlib._bootstrap. Brett Cannon <brett@python.org> parents: 74630 diff changeset	26
ecf4a7bb8807 Relocate importlib._case_ok to importlib._bootstrap. Brett Cannon <brett@python.org> parents: 74630 diff changeset	27
75055 b8576719fe71 _relax_case -> _make_relax_case Antoine Pitrou <solipsis@pitrou.net> parents: 75048 diff changeset	28 def _make_relax_case():
76753 59b8a7fcb047 Properly mark names in importlib._bootstrap as private. Brett Cannon <brett@python.org> parents: 76586 diff changeset	29 if sys.platform.startswith(_CASE_INSENSITIVE_PLATFORMS):
75048 bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function. Antoine Pitrou <solipsis@pitrou.net> parents: 75045 diff changeset	30 def _relax_case():
75070 defd944af91b put docstrings on functions Benjamin Peterson <benjamin@python.org> parents: 75055 diff changeset	31 """True if filenames must be checked case-insensitively."""
75048 bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function. Antoine Pitrou <solipsis@pitrou.net> parents: 75045 diff changeset	32 return b'PYTHONCASEOK' in _os.environ
74643 ecf4a7bb8807 Relocate importlib._case_ok to importlib._bootstrap. Brett Cannon <brett@python.org> parents: 74630 diff changeset	33 else:
75048 bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function. Antoine Pitrou <solipsis@pitrou.net> parents: 75045 diff changeset	34 def _relax_case():
75070 defd944af91b put docstrings on functions Benjamin Peterson <benjamin@python.org> parents: 75055 diff changeset	35 """True if filenames must be checked case-insensitively."""
75048 bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function. Antoine Pitrou <solipsis@pitrou.net> parents: 75045 diff changeset	36 return False
bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function. Antoine Pitrou <solipsis@pitrou.net> parents: 75045 diff changeset	37 return _relax_case
74643 ecf4a7bb8807 Relocate importlib._case_ok to importlib._bootstrap. Brett Cannon <brett@python.org> parents: 74630 diff changeset	38
ecf4a7bb8807 Relocate importlib._case_ok to importlib._bootstrap. Brett Cannon <brett@python.org> parents: 74630 diff changeset	39
74630 7d8f016784cd Move some code from importlib.__init__ to importlib._bootstrap that Brett Cannon <brett@python.org> parents: 74449 diff changeset	40 def _w_long(x):
83963 185a0e649fb8 Issue #18070: importlib.util.module_for_loader() now sets __loader__ Brett Cannon <brett@python.org> parents: 83960 diff changeset	41 """Convert a 32-bit integer to little-endian."""
82385 d98a82f4c9bd Issue #17220: two fixes for changeset 2528e4aea338. Brett Cannon <brett@python.org> parents: 82382 diff changeset	42 return (int(x) & 0xFFFFFFFF).to_bytes(4, 'little')
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	43
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	44
74630 7d8f016784cd Move some code from importlib.__init__ to importlib._bootstrap that Brett Cannon <brett@python.org> parents: 74449 diff changeset	45 def _r_long(int_bytes):
83963 185a0e649fb8 Issue #18070: importlib.util.module_for_loader() now sets __loader__ Brett Cannon <brett@python.org> parents: 83960 diff changeset	46 """Convert 4 bytes in little-endian to an integer."""
82382 2528e4aea338 Issue #17220: Little cleanup of _bootstrap.py. Serhiy Storchaka <storchaka@gmail.com> parents: 82237 diff changeset	47 return int.from_bytes(int_bytes, 'little')
74630 7d8f016784cd Move some code from importlib.__init__ to importlib._bootstrap that Brett Cannon <brett@python.org> parents: 74449 diff changeset	48
7d8f016784cd Move some code from importlib.__init__ to importlib._bootstrap that Brett Cannon <brett@python.org> parents: 74449 diff changeset	49
76472 4e853913054c Issue #13959: Continue to try to accomodate altsep in importlib by not Brett Cannon <brett@python.org> parents: 76466 diff changeset	50 def _path_join(*path_parts):
76449 163360370ad2 Make path manipulation more robust for platforms with alternative path Brett Cannon <brett@python.org> parents: 76445 diff changeset	51 """Replacement for os.path.join()."""
82382 2528e4aea338 Issue #17220: Little cleanup of _bootstrap.py. Serhiy Storchaka <storchaka@gmail.com> parents: 82237 diff changeset	52 return path_sep.join([part.rstrip(path_separators)
2528e4aea338 Issue #17220: Little cleanup of _bootstrap.py. Serhiy Storchaka <storchaka@gmail.com> parents: 82237 diff changeset	53 for part in path_parts if part])
74630 7d8f016784cd Move some code from importlib.__init__ to importlib._bootstrap that Brett Cannon <brett@python.org> parents: 74449 diff changeset	54
76449 163360370ad2 Make path manipulation more robust for platforms with alternative path Brett Cannon <brett@python.org> parents: 76445 diff changeset	55
163360370ad2 Make path manipulation more robust for platforms with alternative path Brett Cannon <brett@python.org> parents: 76445 diff changeset	56 def _path_split(path):
163360370ad2 Make path manipulation more robust for platforms with alternative path Brett Cannon <brett@python.org> parents: 76445 diff changeset	57 """Replacement for os.path.split()."""
82382 2528e4aea338 Issue #17220: Little cleanup of _bootstrap.py. Serhiy Storchaka <storchaka@gmail.com> parents: 82237 diff changeset	58 if len(path_separators) == 1:
2528e4aea338 Issue #17220: Little cleanup of _bootstrap.py. Serhiy Storchaka <storchaka@gmail.com> parents: 82237 diff changeset	59 front, _, tail = path.rpartition(path_sep)
2528e4aea338 Issue #17220: Little cleanup of _bootstrap.py. Serhiy Storchaka <storchaka@gmail.com> parents: 82237 diff changeset	60 return front, tail
76449 163360370ad2 Make path manipulation more robust for platforms with alternative path Brett Cannon <brett@python.org> parents: 76445 diff changeset	61 for x in reversed(path):
163360370ad2 Make path manipulation more robust for platforms with alternative path Brett Cannon <brett@python.org> parents: 76445 diff changeset	62 if x in path_separators:
82385 d98a82f4c9bd Issue #17220: two fixes for changeset 2528e4aea338. Brett Cannon <brett@python.org> parents: 82382 diff changeset	63 front, tail = path.rsplit(x, maxsplit=1)
82382 2528e4aea338 Issue #17220: Little cleanup of _bootstrap.py. Serhiy Storchaka <storchaka@gmail.com> parents: 82237 diff changeset	64 return front, tail
2528e4aea338 Issue #17220: Little cleanup of _bootstrap.py. Serhiy Storchaka <storchaka@gmail.com> parents: 82237 diff changeset	65 return '', path
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	66
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	67
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	68 def _path_is_mode_type(path, mode):
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	69 """Test whether the path is the specified mode type."""
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	70 try:
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	71 stat_info = _os.stat(path)
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	72 except OSError:
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	73 return False
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	74 return (stat_info.st_mode & 0o170000) == mode
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	75
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	76
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	77 # XXX Could also expose Modules/getpath.c:isfile()
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	78 def _path_isfile(path):
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	79 """Replacement for os.path.isfile."""
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	80 return _path_is_mode_type(path, 0o100000)
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	81
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	82
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	83 # XXX Could also expose Modules/getpath.c:isdir()
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	84 def _path_isdir(path):
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	85 """Replacement for os.path.isdir."""
62756 34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	86 if not path:
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	87 path = _os.getcwd()
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	88 return _path_is_mode_type(path, 0o040000)
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	89
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	90
78722 3a831a0a29c4 Close #2501: Permission bits are once again correctly copied from the source file to the cached bytecode file. Test by Eric Snow. Nick Coghlan <ncoghlan@gmail.com> parents: 78663 diff changeset	91 def _write_atomic(path, data, mode=0o666):
75185 27d31f0c4ad5 Issue #14077: importlib: Fix regression introduced by de6703671386. Charles-François Natali <neologix@free.fr> parents: 75082 diff changeset	92 """Best-effort function to write data to a path atomically.
27d31f0c4ad5 Issue #14077: importlib: Fix regression introduced by de6703671386. Charles-François Natali <neologix@free.fr> parents: 75082 diff changeset	93 Be prepared to handle a FileExistsError if concurrent writing of the
27d31f0c4ad5 Issue #14077: importlib: Fix regression introduced by de6703671386. Charles-François Natali <neologix@free.fr> parents: 75082 diff changeset	94 temporary file is attempted."""
73573 b75b41237380 Issue #13392: Writing a pyc file should now be atomic under Windows as well. Antoine Pitrou <solipsis@pitrou.net> parents: 73477 diff changeset	95 # id() is used to generate a pseudo-random filename.
b75b41237380 Issue #13392: Writing a pyc file should now be atomic under Windows as well. Antoine Pitrou <solipsis@pitrou.net> parents: 73477 diff changeset	96 path_tmp = '{}.{}'.format(path, id(path))
78722 3a831a0a29c4 Close #2501: Permission bits are once again correctly copied from the source file to the cached bytecode file. Test by Eric Snow. Nick Coghlan <ncoghlan@gmail.com> parents: 78663 diff changeset	97 fd = _os.open(path_tmp,
3a831a0a29c4 Close #2501: Permission bits are once again correctly copied from the source file to the cached bytecode file. Test by Eric Snow. Nick Coghlan <ncoghlan@gmail.com> parents: 78663 diff changeset	98 _os.O_EXCL \| _os.O_CREAT \| _os.O_WRONLY, mode & 0o666)
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	99 try:
74998 de6703671386 Have importlib use os.replace() for atomic renaming. Brett Cannon <brett@python.org> parents: 74994 diff changeset	100 # We first write data to a temporary file, and then use os.replace() to
de6703671386 Have importlib use os.replace() for atomic renaming. Brett Cannon <brett@python.org> parents: 74994 diff changeset	101 # perform an atomic rename.
73573 b75b41237380 Issue #13392: Writing a pyc file should now be atomic under Windows as well. Antoine Pitrou <solipsis@pitrou.net> parents: 73477 diff changeset	102 with _io.FileIO(fd, 'wb') as file:
b75b41237380 Issue #13392: Writing a pyc file should now be atomic under Windows as well. Antoine Pitrou <solipsis@pitrou.net> parents: 73477 diff changeset	103 file.write(data)
74998 de6703671386 Have importlib use os.replace() for atomic renaming. Brett Cannon <brett@python.org> parents: 74994 diff changeset	104 _os.replace(path_tmp, path)
73573 b75b41237380 Issue #13392: Writing a pyc file should now be atomic under Windows as well. Antoine Pitrou <solipsis@pitrou.net> parents: 73477 diff changeset	105 except OSError:
b75b41237380 Issue #13392: Writing a pyc file should now be atomic under Windows as well. Antoine Pitrou <solipsis@pitrou.net> parents: 73477 diff changeset	106 try:
b75b41237380 Issue #13392: Writing a pyc file should now be atomic under Windows as well. Antoine Pitrou <solipsis@pitrou.net> parents: 73477 diff changeset	107 _os.unlink(path_tmp)
72960 c16063765d3a Issue #13146: Writing a pyc file is now atomic under POSIX. Antoine Pitrou <solipsis@pitrou.net> parents: 68887 diff changeset	108 except OSError:
73573 b75b41237380 Issue #13392: Writing a pyc file should now be atomic under Windows as well. Antoine Pitrou <solipsis@pitrou.net> parents: 73477 diff changeset	109 pass
b75b41237380 Issue #13392: Writing a pyc file should now be atomic under Windows as well. Antoine Pitrou <solipsis@pitrou.net> parents: 73477 diff changeset	110 raise
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	111
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	112
52139 2e8e897a1244 Make utility code in importlib._bootstrap private. Brett Cannon <bcannon@gmail.com> parents: 52115 diff changeset	113 def _wrap(new, old):
77964 37850f083af5 Minor refactoring in importlib._bootstrap, and fix the '_wrap' docstring. Florent Xicluna <florent.xicluna@gmail.com> parents: 77917 diff changeset	114 """Simple substitute for functools.update_wrapper."""
73977 54a77c556d9a Issue #13593: updating the importlib utility decorators for __qualname__. Meador Inge <meadori@gmail.com> parents: 73974 diff changeset	115 for replace in ['__module__', '__name__', '__qualname__', '__doc__']:
74838 7a6fd7cd16e9 Don't fail in the face of a lacking attribute when wrapping a Brett Cannon <brett@python.org> parents: 74643 diff changeset	116 if hasattr(old, replace):
7a6fd7cd16e9 Don't fail in the face of a lacking attribute when wrapping a Brett Cannon <brett@python.org> parents: 74643 diff changeset	117 setattr(new, replace, getattr(old, replace))
51433 43d3a95ed662 Create a simple substitute for functools.wraps to use in importlib._bootstrap. Brett Cannon <bcannon@gmail.com> parents: 51430 diff changeset	118 new.__dict__.update(old.__dict__)
43d3a95ed662 Create a simple substitute for functools.wraps to use in importlib._bootstrap. Brett Cannon <bcannon@gmail.com> parents: 51430 diff changeset	119
43d3a95ed662 Create a simple substitute for functools.wraps to use in importlib._bootstrap. Brett Cannon <bcannon@gmail.com> parents: 51430 diff changeset	120
76753 59b8a7fcb047 Properly mark names in importlib._bootstrap as private. Brett Cannon <brett@python.org> parents: 76586 diff changeset	121 _code_type = type(_wrap.__code__)
62756 34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	122
76331 d777f854a66e Issue #13959: Rename imp to _imp and add Lib/imp.py and begin Brett Cannon <brett@python.org> parents: 76330 diff changeset	123
77013 edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	124 # Module-level locking ########################################################
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	125
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	126 # A dict mapping module names to weakrefs of _ModuleLock instances
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	127 _module_locks = {}
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	128 # A dict mapping thread ids to _ModuleLock instances
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	129 _blocking_on = {}
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	130
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	131
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	132 class _DeadlockError(RuntimeError):
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	133 pass
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	134
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	135
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	136 class _ModuleLock:
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	137 """A recursive lock implementation which is able to detect deadlocks
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	138 (e.g. thread 1 trying to take locks A then B, and thread 2 trying to
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	139 take locks B then A).
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	140 """
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	141
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	142 def __init__(self, name):
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	143 self.lock = _thread.allocate_lock()
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	144 self.wakeup = _thread.allocate_lock()
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	145 self.name = name
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	146 self.owner = None
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	147 self.count = 0
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	148 self.waiters = 0
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	149
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	150 def has_deadlock(self):
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	151 # Deadlock avoidance for concurrent circular imports.
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	152 me = _thread.get_ident()
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	153 tid = self.owner
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	154 while True:
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	155 lock = _blocking_on.get(tid)
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	156 if lock is None:
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	157 return False
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	158 tid = lock.owner
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	159 if tid == me:
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	160 return True
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	161
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	162 def acquire(self):
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	163 """
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	164 Acquire the module lock. If a potential deadlock is detected,
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	165 a _DeadlockError is raised.
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	166 Otherwise, the lock is always acquired and True is returned.
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	167 """
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	168 tid = _thread.get_ident()
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	169 _blocking_on[tid] = self
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	170 try:
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	171 while True:
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	172 with self.lock:
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	173 if self.count == 0 or self.owner == tid:
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	174 self.owner = tid
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	175 self.count += 1
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	176 return True
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	177 if self.has_deadlock():
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	178 raise _DeadlockError("deadlock detected by %r" % self)
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	179 if self.wakeup.acquire(False):
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	180 self.waiters += 1
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	181 # Wait for a release() call
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	182 self.wakeup.acquire()
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	183 self.wakeup.release()
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	184 finally:
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	185 del _blocking_on[tid]
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	186
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	187 def release(self):
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	188 tid = _thread.get_ident()
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	189 with self.lock:
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	190 if self.owner != tid:
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	191 raise RuntimeError("cannot release un-acquired lock")
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	192 assert self.count > 0
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	193 self.count -= 1
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	194 if self.count == 0:
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	195 self.owner = None
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	196 if self.waiters:
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	197 self.waiters -= 1
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	198 self.wakeup.release()
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	199
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	200 def __repr__(self):
79692 987a8724b220 Remove uses of % with str.format. Brett Cannon <brett@python.org> parents: 79653 diff changeset	201 return "_ModuleLock({!r}) at {}".format(self.name, id(self))
77013 edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	202
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	203
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	204 class _DummyModuleLock:
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	205 """A simple _ModuleLock equivalent for Python builds without
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	206 multi-threading support."""
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	207
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	208 def __init__(self, name):
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	209 self.name = name
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	210 self.count = 0
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	211
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	212 def acquire(self):
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	213 self.count += 1
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	214 return True
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	215
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	216 def release(self):
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	217 if self.count == 0:
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	218 raise RuntimeError("cannot release un-acquired lock")
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	219 self.count -= 1
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	220
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	221 def __repr__(self):
79692 987a8724b220 Remove uses of % with str.format. Brett Cannon <brett@python.org> parents: 79653 diff changeset	222 return "_DummyModuleLock({!r}) at {}".format(self.name, id(self))
77013 edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	223
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	224
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	225 # The following two functions are for consumption by Python/import.c.
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	226
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	227 def _get_module_lock(name):
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	228 """Get or create the module lock for a given module name.
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	229
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	230 Should only be called with the import lock taken."""
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	231 lock = None
78779 7fa6336e9864 Issue #15781: Fix two small race conditions in import's module locking. Antoine Pitrou <solipsis@pitrou.net> parents: 78758 diff changeset	232 try:
77013 edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	233 lock = _module_locks[name]()
78779 7fa6336e9864 Issue #15781: Fix two small race conditions in import's module locking. Antoine Pitrou <solipsis@pitrou.net> parents: 78758 diff changeset	234 except KeyError:
7fa6336e9864 Issue #15781: Fix two small race conditions in import's module locking. Antoine Pitrou <solipsis@pitrou.net> parents: 78758 diff changeset	235 pass
77013 edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	236 if lock is None:
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	237 if _thread is None:
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	238 lock = _DummyModuleLock(name)
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	239 else:
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	240 lock = _ModuleLock(name)
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	241 def cb(_):
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	242 del _module_locks[name]
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	243 _module_locks[name] = _weakref.ref(lock, cb)
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	244 return lock
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	245
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	246 def _lock_unlock_module(name):
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	247 """Release the global import lock, and acquires then release the
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	248 module lock for a given module name.
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	249 This is used to ensure a module is completely initialized, in the
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	250 event it is being imported by another thread.
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	251
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	252 Should only be called with the import lock taken."""
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	253 lock = _get_module_lock(name)
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	254 _imp.release_lock()
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	255 try:
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	256 lock.acquire()
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	257 except _DeadlockError:
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	258 # Concurrent circular import, we'll accept a partially initialized
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	259 # module object.
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	260 pass
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	261 else:
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	262 lock.release()
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	263
78344 62033490ca0f Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise Nick Coghlan <ncoghlan@gmail.com> parents: 78314 diff changeset	264 # Frame stripping magic ###############################################
77013 edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	265
78344 62033490ca0f Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise Nick Coghlan <ncoghlan@gmail.com> parents: 78314 diff changeset	266 def _call_with_frames_removed(f, args, *kwds):
62033490ca0f Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise Nick Coghlan <ncoghlan@gmail.com> parents: 78314 diff changeset	267 """remove_importlib_frames in import.c will always remove sequences
62033490ca0f Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise Nick Coghlan <ncoghlan@gmail.com> parents: 78314 diff changeset	268 of importlib frames that end with a call to this function
62033490ca0f Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise Nick Coghlan <ncoghlan@gmail.com> parents: 78314 diff changeset	269
62033490ca0f Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise Nick Coghlan <ncoghlan@gmail.com> parents: 78314 diff changeset	270 Use it instead of a normal call in places where including the importlib
62033490ca0f Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise Nick Coghlan <ncoghlan@gmail.com> parents: 78314 diff changeset	271 frames introduces unwanted noise into the traceback (e.g. when executing
62033490ca0f Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise Nick Coghlan <ncoghlan@gmail.com> parents: 78314 diff changeset	272 module code)
62033490ca0f Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise Nick Coghlan <ncoghlan@gmail.com> parents: 78314 diff changeset	273 """
62033490ca0f Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise Nick Coghlan <ncoghlan@gmail.com> parents: 78314 diff changeset	274 return f(args, *kwds)
62033490ca0f Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise Nick Coghlan <ncoghlan@gmail.com> parents: 78314 diff changeset	275
62033490ca0f Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise Nick Coghlan <ncoghlan@gmail.com> parents: 78314 diff changeset	276
62033490ca0f Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise Nick Coghlan <ncoghlan@gmail.com> parents: 78314 diff changeset	277 # Finder/loader utility code ###############################################
52140 1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap. Brett Cannon <bcannon@gmail.com> parents: 52139 diff changeset	278
82957 313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	279 # Magic word to reject .pyc files generated by other Python versions.
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	280 # It should change for each incompatible change to the bytecode.
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	281 #
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	282 # The value of CR and LF is incorporated so if you ever read or write
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	283 # a .pyc file in text mode the magic number will be wrong; also, the
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	284 # Apple MPW compiler swaps their values, botching string constants.
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	285 #
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	286 # The magic numbers must be spaced apart at least 2 values, as the
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	287 # -U interpeter flag will cause MAGIC+1 being used. They have been
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	288 # odd numbers for some time now.
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	289 #
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	290 # There were a variety of old schemes for setting the magic number.
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	291 # The current working scheme is to increment the previous value by
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	292 # 10.
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	293 #
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	294 # Starting with the adoption of PEP 3147 in Python 3.2, every bump in magic
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	295 # number also includes a new "magic tag", i.e. a human readable string used
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	296 # to represent the magic number in __pycache__ directories. When you change
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	297 # the magic number, you must also set a new unique magic tag. Generally this
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	298 # can be named after the Python major version of the magic number bump, but
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	299 # it can really be anything, as long as it's different than anything else
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	300 # that's come before. The tags are included in the following table, starting
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	301 # with Python 3.2a0.
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	302 #
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	303 # Known values:
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	304 # Python 1.5: 20121
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	305 # Python 1.5.1: 20121
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	306 # Python 1.5.2: 20121
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	307 # Python 1.6: 50428
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	308 # Python 2.0: 50823
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	309 # Python 2.0.1: 50823
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	310 # Python 2.1: 60202
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	311 # Python 2.1.1: 60202
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	312 # Python 2.1.2: 60202
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	313 # Python 2.2: 60717
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	314 # Python 2.3a0: 62011
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	315 # Python 2.3a0: 62021
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	316 # Python 2.3a0: 62011 (!)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	317 # Python 2.4a0: 62041
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	318 # Python 2.4a3: 62051
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	319 # Python 2.4b1: 62061
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	320 # Python 2.5a0: 62071
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	321 # Python 2.5a0: 62081 (ast-branch)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	322 # Python 2.5a0: 62091 (with)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	323 # Python 2.5a0: 62092 (changed WITH_CLEANUP opcode)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	324 # Python 2.5b3: 62101 (fix wrong code: for x, in ...)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	325 # Python 2.5b3: 62111 (fix wrong code: x += yield)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	326 # Python 2.5c1: 62121 (fix wrong lnotab with for loops and
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	327 # storing constants that should have been removed)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	328 # Python 2.5c2: 62131 (fix wrong code: for x, in ... in listcomp/genexp)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	329 # Python 2.6a0: 62151 (peephole optimizations and STORE_MAP opcode)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	330 # Python 2.6a1: 62161 (WITH_CLEANUP optimization)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	331 # Python 2.7a0: 62171 (optimize list comprehensions/change LIST_APPEND)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	332 # Python 2.7a0: 62181 (optimize conditional branches:
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	333 # introduce POP_JUMP_IF_FALSE and POP_JUMP_IF_TRUE)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	334 # Python 2.7a0 62191 (introduce SETUP_WITH)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	335 # Python 2.7a0 62201 (introduce BUILD_SET)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	336 # Python 2.7a0 62211 (introduce MAP_ADD and SET_ADD)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	337 # Python 3000: 3000
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	338 # 3010 (removed UNARY_CONVERT)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	339 # 3020 (added BUILD_SET)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	340 # 3030 (added keyword-only parameters)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	341 # 3040 (added signature annotations)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	342 # 3050 (print becomes a function)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	343 # 3060 (PEP 3115 metaclass syntax)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	344 # 3061 (string literals become unicode)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	345 # 3071 (PEP 3109 raise changes)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	346 # 3081 (PEP 3137 make __file__ and __name__ unicode)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	347 # 3091 (kill str8 interning)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	348 # 3101 (merge from 2.6a0, see 62151)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	349 # 3103 (__file__ points to source file)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	350 # Python 3.0a4: 3111 (WITH_CLEANUP optimization).
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	351 # Python 3.0a5: 3131 (lexical exception stacking, including POP_EXCEPT)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	352 # Python 3.1a0: 3141 (optimize list, set and dict comprehensions:
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	353 # change LIST_APPEND and SET_ADD, add MAP_ADD)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	354 # Python 3.1a0: 3151 (optimize conditional branches:
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	355 # introduce POP_JUMP_IF_FALSE and POP_JUMP_IF_TRUE)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	356 # Python 3.2a0: 3160 (add SETUP_WITH)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	357 # tag: cpython-32
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	358 # Python 3.2a1: 3170 (add DUP_TOP_TWO, remove DUP_TOPX and ROT_FOUR)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	359 # tag: cpython-32
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	360 # Python 3.2a2 3180 (add DELETE_DEREF)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	361 # Python 3.3a0 3190 __class__ super closure changed
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	362 # Python 3.3a0 3200 (__qualname__ added)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	363 # 3210 (added size modulo 2**32 to the pyc header)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	364 # Python 3.3a1 3220 (changed PEP 380 implementation)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	365 # Python 3.3a4 3230 (revert changes to implicit __class__ closure)
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	366 # Python 3.4a1 3250 (evaluate positional default arguments before
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	367 # keyword-only defaults)
83572 cf65c7a75f55 check local class namespace before reaching for cells (closes #17853) Benjamin Peterson <benjamin@python.org> parents: 83376 diff changeset	368 # Python 3.4a1 3260 (add LOAD_CLASSDEREF; allow locals of class to override
cf65c7a75f55 check local class namespace before reaching for cells (closes #17853) Benjamin Peterson <benjamin@python.org> parents: 83376 diff changeset	369 # free vars)
84090 32527aa9ad16 typo fix Brett Cannon <brett@python.org> parents: 84046 diff changeset	370 # Python 3.4a1 3270 (various tweaks to the __class__ closure)
83791 6db3741e59be rather than passing locals to the class body, just execute the class body in the proper environment Benjamin Peterson <benjamin@python.org> parents: 83783 diff changeset	371 # Python 3.4a1 3280 (remove implicit class argument)
82957 313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	372 #
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	373 # MAGIC must change whenever the bytecode emitted by the compiler may no
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	374 # longer be understood by older implementations of the eval loop (usually
313bcff51900 Issue #17516: use comment syntax for comments, instead of multiline string Victor Stinner <victor.stinner@gmail.com> parents: 82876 diff changeset	375 # due to the addition of new opcodes).
78042 efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	376
84128 5619bc2d8207 Issue #18192: Introduce importlib.util.MAGIC_NUMBER and document the Brett Cannon <brett@python.org> parents: 84109 diff changeset	377 MAGIC_NUMBER = (3280).to_bytes(2, 'little') + b'\r\n'
5619bc2d8207 Issue #18192: Introduce importlib.util.MAGIC_NUMBER and document the Brett Cannon <brett@python.org> parents: 84109 diff changeset	378 _RAW_MAGIC_NUMBER = int.from_bytes(MAGIC_NUMBER, 'little') # For import.c
78042 efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	379
76753 59b8a7fcb047 Properly mark names in importlib._bootstrap as private. Brett Cannon <brett@python.org> parents: 76586 diff changeset	380 _PYCACHE = '__pycache__'
76450 b773a751c2e7 Issue #13959: Re-implement imp.cache_from_source() in Lib/imp.py. Brett Cannon <brett@python.org> parents: 76449 diff changeset	381
76874 b81ddaf0db47 Issue #13959: Deprecate imp.get_suffixes() for new attributes on Brett Cannon <brett@python.org> parents: 76826 diff changeset	382 SOURCE_SUFFIXES = ['.py'] # _setup() adds .pyw as needed.
76451 ea46ebba8a0f Issue #13959: Re-implement imp.source_from_cache() in Lib/imp.py. Brett Cannon <brett@python.org> parents: 76450 diff changeset	383
76874 b81ddaf0db47 Issue #13959: Deprecate imp.get_suffixes() for new attributes on Brett Cannon <brett@python.org> parents: 76826 diff changeset	384 DEBUG_BYTECODE_SUFFIXES = ['.pyc']
b81ddaf0db47 Issue #13959: Deprecate imp.get_suffixes() for new attributes on Brett Cannon <brett@python.org> parents: 76826 diff changeset	385 OPTIMIZED_BYTECODE_SUFFIXES = ['.pyo']
76450 b773a751c2e7 Issue #13959: Re-implement imp.cache_from_source() in Lib/imp.py. Brett Cannon <brett@python.org> parents: 76449 diff changeset	386
76753 59b8a7fcb047 Properly mark names in importlib._bootstrap as private. Brett Cannon <brett@python.org> parents: 76586 diff changeset	387 def cache_from_source(path, debug_override=None):
76450 b773a751c2e7 Issue #13959: Re-implement imp.cache_from_source() in Lib/imp.py. Brett Cannon <brett@python.org> parents: 76449 diff changeset	388 """Given the path to a .py file, return the path to its .pyc/.pyo file.
b773a751c2e7 Issue #13959: Re-implement imp.cache_from_source() in Lib/imp.py. Brett Cannon <brett@python.org> parents: 76449 diff changeset	389
b773a751c2e7 Issue #13959: Re-implement imp.cache_from_source() in Lib/imp.py. Brett Cannon <brett@python.org> parents: 76449 diff changeset	390 The .py file does not need to exist; this simply returns the path to the
b773a751c2e7 Issue #13959: Re-implement imp.cache_from_source() in Lib/imp.py. Brett Cannon <brett@python.org> parents: 76449 diff changeset	391 .pyc/.pyo file calculated as if the .py file were imported. The extension
79173 4de5e4ec3cff don't depend on __debug__ because it's baked in at freeze time (issue #16046) Benjamin Peterson <benjamin@python.org> parents: 78779 diff changeset	392 will be .pyc unless sys.flags.optimize is non-zero, then it will be .pyo.
76450 b773a751c2e7 Issue #13959: Re-implement imp.cache_from_source() in Lib/imp.py. Brett Cannon <brett@python.org> parents: 76449 diff changeset	393
79191 9b0582477aad rephrase Benjamin Peterson <benjamin@python.org> parents: 79173 diff changeset	394 If debug_override is not None, then it must be a boolean and is used in
9b0582477aad rephrase Benjamin Peterson <benjamin@python.org> parents: 79173 diff changeset	395 place of sys.flags.optimize.
76450 b773a751c2e7 Issue #13959: Re-implement imp.cache_from_source() in Lib/imp.py. Brett Cannon <brett@python.org> parents: 76449 diff changeset	396
78026 e86330669bb5 Issue #15056: imp.cache_from_source() and source_from_cache() raise Brett Cannon <brett@python.org> parents: 78010 diff changeset	397 If sys.implementation.cache_tag is None then NotImplementedError is raised.
e86330669bb5 Issue #15056: imp.cache_from_source() and source_from_cache() raise Brett Cannon <brett@python.org> parents: 78010 diff changeset	398
76450 b773a751c2e7 Issue #13959: Re-implement imp.cache_from_source() in Lib/imp.py. Brett Cannon <brett@python.org> parents: 76449 diff changeset	399 """
79173 4de5e4ec3cff don't depend on __debug__ because it's baked in at freeze time (issue #16046) Benjamin Peterson <benjamin@python.org> parents: 78779 diff changeset	400 debug = not sys.flags.optimize if debug_override is None else debug_override
76874 b81ddaf0db47 Issue #13959: Deprecate imp.get_suffixes() for new attributes on Brett Cannon <brett@python.org> parents: 76826 diff changeset	401 if debug:
b81ddaf0db47 Issue #13959: Deprecate imp.get_suffixes() for new attributes on Brett Cannon <brett@python.org> parents: 76826 diff changeset	402 suffixes = DEBUG_BYTECODE_SUFFIXES
b81ddaf0db47 Issue #13959: Deprecate imp.get_suffixes() for new attributes on Brett Cannon <brett@python.org> parents: 76826 diff changeset	403 else:
b81ddaf0db47 Issue #13959: Deprecate imp.get_suffixes() for new attributes on Brett Cannon <brett@python.org> parents: 76826 diff changeset	404 suffixes = OPTIMIZED_BYTECODE_SUFFIXES
76450 b773a751c2e7 Issue #13959: Re-implement imp.cache_from_source() in Lib/imp.py. Brett Cannon <brett@python.org> parents: 76449 diff changeset	405 head, tail = _path_split(path)
b773a751c2e7 Issue #13959: Re-implement imp.cache_from_source() in Lib/imp.py. Brett Cannon <brett@python.org> parents: 76449 diff changeset	406 base_filename, sep, _ = tail.partition('.')
78026 e86330669bb5 Issue #15056: imp.cache_from_source() and source_from_cache() raise Brett Cannon <brett@python.org> parents: 78010 diff changeset	407 tag = sys.implementation.cache_tag
e86330669bb5 Issue #15056: imp.cache_from_source() and source_from_cache() raise Brett Cannon <brett@python.org> parents: 78010 diff changeset	408 if tag is None:
e86330669bb5 Issue #15056: imp.cache_from_source() and source_from_cache() raise Brett Cannon <brett@python.org> parents: 78010 diff changeset	409 raise NotImplementedError('sys.implementation.cache_tag is None')
e86330669bb5 Issue #15056: imp.cache_from_source() and source_from_cache() raise Brett Cannon <brett@python.org> parents: 78010 diff changeset	410 filename = ''.join([base_filename, sep, tag, suffixes[0]])
76753 59b8a7fcb047 Properly mark names in importlib._bootstrap as private. Brett Cannon <brett@python.org> parents: 76586 diff changeset	411 return _path_join(head, _PYCACHE, filename)
76450 b773a751c2e7 Issue #13959: Re-implement imp.cache_from_source() in Lib/imp.py. Brett Cannon <brett@python.org> parents: 76449 diff changeset	412
b773a751c2e7 Issue #13959: Re-implement imp.cache_from_source() in Lib/imp.py. Brett Cannon <brett@python.org> parents: 76449 diff changeset	413
78078 9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	414 def source_from_cache(path):
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	415 """Given the path to a .pyc./.pyo file, return the path to its .py file.
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	416
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	417 The .pyc/.pyo file does not need to exist; this simply returns the path to
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	418 the .py file calculated to correspond to the .pyc/.pyo file. If path does
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	419 not conform to PEP 3147 format, ValueError will be raised. If
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	420 sys.implementation.cache_tag is None then NotImplementedError is raised.
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	421
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	422 """
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	423 if sys.implementation.cache_tag is None:
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	424 raise NotImplementedError('sys.implementation.cache_tag is None')
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	425 head, pycache_filename = _path_split(path)
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	426 head, pycache = _path_split(head)
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	427 if pycache != _PYCACHE:
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	428 raise ValueError('{} not bottom-level directory in '
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	429 '{!r}'.format(_PYCACHE, path))
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	430 if pycache_filename.count('.') != 2:
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	431 raise ValueError('expected only 2 dots in '
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	432 '{!r}'.format(pycache_filename))
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	433 base_filename = pycache_filename.partition('.')[0]
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	434 return _path_join(head, base_filename + SOURCE_SUFFIXES[0])
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	435
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	436
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	437 def _get_sourcefile(bytecode_path):
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	438 """Convert a bytecode file path to a source path (if possible).
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	439
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	440 This function exists purely for backwards-compatibility for
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	441 PyImport_ExecCodeModuleWithFilenames() in the C API.
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	442
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	443 """
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	444 if len(bytecode_path) == 0:
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	445 return None
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	446 rest, _, extension = bytecode_path.rparition('.')
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	447 if not rest or extension.lower()[-3:-1] != '.py':
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	448 return bytecode_path
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	449
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	450 try:
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	451 source_path = source_from_cache(bytecode_path)
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	452 except (NotImplementedError, ValueError):
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	453 source_path = bytcode_path[-1:]
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	454
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	455 return source_path if _path_isfile(source_stats) else bytecode_path
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	456
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	457
83376 04aaeae6ee7b Issue #17244: Don't mask exceptions raised during the creation of Brett Cannon <brett@python.org> parents: 83225 diff changeset	458 def _calc_mode(path):
04aaeae6ee7b Issue #17244: Don't mask exceptions raised during the creation of Brett Cannon <brett@python.org> parents: 83225 diff changeset	459 """Calculate the mode permissions for a bytecode file."""
04aaeae6ee7b Issue #17244: Don't mask exceptions raised during the creation of Brett Cannon <brett@python.org> parents: 83225 diff changeset	460 try:
04aaeae6ee7b Issue #17244: Don't mask exceptions raised during the creation of Brett Cannon <brett@python.org> parents: 83225 diff changeset	461 mode = _os.stat(path).st_mode
04aaeae6ee7b Issue #17244: Don't mask exceptions raised during the creation of Brett Cannon <brett@python.org> parents: 83225 diff changeset	462 except OSError:
04aaeae6ee7b Issue #17244: Don't mask exceptions raised during the creation of Brett Cannon <brett@python.org> parents: 83225 diff changeset	463 mode = 0o666
04aaeae6ee7b Issue #17244: Don't mask exceptions raised during the creation of Brett Cannon <brett@python.org> parents: 83225 diff changeset	464 # We always ensure write access so we can update cached files
04aaeae6ee7b Issue #17244: Don't mask exceptions raised during the creation of Brett Cannon <brett@python.org> parents: 83225 diff changeset	465 # later even when the source files are read-only on Windows (#6074)
04aaeae6ee7b Issue #17244: Don't mask exceptions raised during the creation of Brett Cannon <brett@python.org> parents: 83225 diff changeset	466 mode \|= 0o200
04aaeae6ee7b Issue #17244: Don't mask exceptions raised during the creation of Brett Cannon <brett@python.org> parents: 83225 diff changeset	467 return mode
04aaeae6ee7b Issue #17244: Don't mask exceptions raised during the creation of Brett Cannon <brett@python.org> parents: 83225 diff changeset	468
04aaeae6ee7b Issue #17244: Don't mask exceptions raised during the creation of Brett Cannon <brett@python.org> parents: 83225 diff changeset	469
83055 ac89e7bc0e2f Issue #17357: Add missing verbosity messages when running under Brett Cannon <brett@python.org> parents: 82236 diff changeset	470 def _verbose_message(message, *args, verbosity=1):
76303 2dd046be2c88 Issue #2377: Make importlib the implementation of __import__(). Brett Cannon <brett@python.org> parents: 76280 diff changeset	471 """Print the message to stderr if -v/PYTHONVERBOSE is turned on."""
83055 ac89e7bc0e2f Issue #17357: Add missing verbosity messages when running under Brett Cannon <brett@python.org> parents: 82236 diff changeset	472 if sys.flags.verbose >= verbosity:
76327 1ca10eb58483 utilize startswith(tuple) Philip Jenvey <pjenvey@underboss.org> parents: 76312 diff changeset	473 if not message.startswith(('#', 'import ')):
76303 2dd046be2c88 Issue #2377: Make importlib the implementation of __import__(). Brett Cannon <brett@python.org> parents: 76280 diff changeset	474 message = '# ' + message
2dd046be2c88 Issue #2377: Make importlib the implementation of __import__(). Brett Cannon <brett@python.org> parents: 76280 diff changeset	475 print(message.format(*args), file=sys.stderr)
2dd046be2c88 Issue #2377: Make importlib the implementation of __import__(). Brett Cannon <brett@python.org> parents: 76280 diff changeset	476
52140 1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap. Brett Cannon <bcannon@gmail.com> parents: 52139 diff changeset	477
84171 205aa49e5cd5 Issue #18115: Abstract out managing the cleanup of modules to use in Brett Cannon <brett@python.org> parents: 84169 diff changeset	478 class _ManageReload:
205aa49e5cd5 Issue #18115: Abstract out managing the cleanup of modules to use in Brett Cannon <brett@python.org> parents: 84169 diff changeset	479
205aa49e5cd5 Issue #18115: Abstract out managing the cleanup of modules to use in Brett Cannon <brett@python.org> parents: 84169 diff changeset	480 def __init__(self, name):
205aa49e5cd5 Issue #18115: Abstract out managing the cleanup of modules to use in Brett Cannon <brett@python.org> parents: 84169 diff changeset	481 self._name = name
205aa49e5cd5 Issue #18115: Abstract out managing the cleanup of modules to use in Brett Cannon <brett@python.org> parents: 84169 diff changeset	482
205aa49e5cd5 Issue #18115: Abstract out managing the cleanup of modules to use in Brett Cannon <brett@python.org> parents: 84169 diff changeset	483 def __enter__(self):
205aa49e5cd5 Issue #18115: Abstract out managing the cleanup of modules to use in Brett Cannon <brett@python.org> parents: 84169 diff changeset	484 self._is_reload = self._name in sys.modules
205aa49e5cd5 Issue #18115: Abstract out managing the cleanup of modules to use in Brett Cannon <brett@python.org> parents: 84169 diff changeset	485
205aa49e5cd5 Issue #18115: Abstract out managing the cleanup of modules to use in Brett Cannon <brett@python.org> parents: 84169 diff changeset	486 def __exit__(self, *args):
205aa49e5cd5 Issue #18115: Abstract out managing the cleanup of modules to use in Brett Cannon <brett@python.org> parents: 84169 diff changeset	487 if any(arg is not None for arg in args) and not self._is_reload:
205aa49e5cd5 Issue #18115: Abstract out managing the cleanup of modules to use in Brett Cannon <brett@python.org> parents: 84169 diff changeset	488 try:
205aa49e5cd5 Issue #18115: Abstract out managing the cleanup of modules to use in Brett Cannon <brett@python.org> parents: 84169 diff changeset	489 del sys.modules[self._name]
205aa49e5cd5 Issue #18115: Abstract out managing the cleanup of modules to use in Brett Cannon <brett@python.org> parents: 84169 diff changeset	490 except KeyError:
205aa49e5cd5 Issue #18115: Abstract out managing the cleanup of modules to use in Brett Cannon <brett@python.org> parents: 84169 diff changeset	491 pass
205aa49e5cd5 Issue #18115: Abstract out managing the cleanup of modules to use in Brett Cannon <brett@python.org> parents: 84169 diff changeset	492
205aa49e5cd5 Issue #18115: Abstract out managing the cleanup of modules to use in Brett Cannon <brett@python.org> parents: 84169 diff changeset	493
83988 6ace4133c1c0 Rename importlib.util.ModuleManager to module_to_load so that the name Brett Cannon <brett@python.org> parents: 83963 diff changeset	494 # Written as a class only because contextlib is not available.
84171 205aa49e5cd5 Issue #18115: Abstract out managing the cleanup of modules to use in Brett Cannon <brett@python.org> parents: 84169 diff changeset	495 class _ModuleManager(_ManageReload):
83960 3a1976680717 Introduce importlib.util.ModuleManager which is a context manager to Brett Cannon <brett@python.org> parents: 83791 diff changeset	496
3a1976680717 Introduce importlib.util.ModuleManager which is a context manager to Brett Cannon <brett@python.org> parents: 83791 diff changeset	497 """Context manager which returns the module to be loaded.
3a1976680717 Introduce importlib.util.ModuleManager which is a context manager to Brett Cannon <brett@python.org> parents: 83791 diff changeset	498
3a1976680717 Introduce importlib.util.ModuleManager which is a context manager to Brett Cannon <brett@python.org> parents: 83791 diff changeset	499 Does the proper unloading from sys.modules upon failure.
3a1976680717 Introduce importlib.util.ModuleManager which is a context manager to Brett Cannon <brett@python.org> parents: 83791 diff changeset	500
3a1976680717 Introduce importlib.util.ModuleManager which is a context manager to Brett Cannon <brett@python.org> parents: 83791 diff changeset	501 """
3a1976680717 Introduce importlib.util.ModuleManager which is a context manager to Brett Cannon <brett@python.org> parents: 83791 diff changeset	502
83998 39cc1b04713e Add a reset_name argument to importlib.util.module_to_load in order to Brett Cannon <brett@python.org> parents: 83988 diff changeset	503 def __init__(self, name, *, reset_name=True):
39cc1b04713e Add a reset_name argument to importlib.util.module_to_load in order to Brett Cannon <brett@python.org> parents: 83988 diff changeset	504 """Prepare the context manager.
39cc1b04713e Add a reset_name argument to importlib.util.module_to_load in order to Brett Cannon <brett@python.org> parents: 83988 diff changeset	505
39cc1b04713e Add a reset_name argument to importlib.util.module_to_load in order to Brett Cannon <brett@python.org> parents: 83988 diff changeset	506 The reset_name argument specifies whether to unconditionally reset
39cc1b04713e Add a reset_name argument to importlib.util.module_to_load in order to Brett Cannon <brett@python.org> parents: 83988 diff changeset	507 the __name__ attribute if the module is found to be a reload.
39cc1b04713e Add a reset_name argument to importlib.util.module_to_load in order to Brett Cannon <brett@python.org> parents: 83988 diff changeset	508 """
84171 205aa49e5cd5 Issue #18115: Abstract out managing the cleanup of modules to use in Brett Cannon <brett@python.org> parents: 84169 diff changeset	509 super().__init__(name)
83998 39cc1b04713e Add a reset_name argument to importlib.util.module_to_load in order to Brett Cannon <brett@python.org> parents: 83988 diff changeset	510 self._reset_name = reset_name
83960 3a1976680717 Introduce importlib.util.ModuleManager which is a context manager to Brett Cannon <brett@python.org> parents: 83791 diff changeset	511
3a1976680717 Introduce importlib.util.ModuleManager which is a context manager to Brett Cannon <brett@python.org> parents: 83791 diff changeset	512 def __enter__(self):
84171 205aa49e5cd5 Issue #18115: Abstract out managing the cleanup of modules to use in Brett Cannon <brett@python.org> parents: 84169 diff changeset	513 super().__enter__()
83960 3a1976680717 Introduce importlib.util.ModuleManager which is a context manager to Brett Cannon <brett@python.org> parents: 83791 diff changeset	514 self._module = sys.modules.get(self._name)
3a1976680717 Introduce importlib.util.ModuleManager which is a context manager to Brett Cannon <brett@python.org> parents: 83791 diff changeset	515 if not self._is_reload:
3a1976680717 Introduce importlib.util.ModuleManager which is a context manager to Brett Cannon <brett@python.org> parents: 83791 diff changeset	516 # This must be done before open() is called as the 'io' module
3a1976680717 Introduce importlib.util.ModuleManager which is a context manager to Brett Cannon <brett@python.org> parents: 83791 diff changeset	517 # implicitly imports 'locale' and would otherwise trigger an
3a1976680717 Introduce importlib.util.ModuleManager which is a context manager to Brett Cannon <brett@python.org> parents: 83791 diff changeset	518 # infinite loop.
84134 9cacdb9d0c59 Issue #17907: touch up the code for imp.new_module(). Brett Cannon <brett@python.org> parents: 84128 diff changeset	519 self._module = type(_io)(self._name)
83960 3a1976680717 Introduce importlib.util.ModuleManager which is a context manager to Brett Cannon <brett@python.org> parents: 83791 diff changeset	520 # This must be done before putting the module in sys.modules
3a1976680717 Introduce importlib.util.ModuleManager which is a context manager to Brett Cannon <brett@python.org> parents: 83791 diff changeset	521 # (otherwise an optimization shortcut in import.c becomes wrong)
3a1976680717 Introduce importlib.util.ModuleManager which is a context manager to Brett Cannon <brett@python.org> parents: 83791 diff changeset	522 self._module.__initializing__ = True
3a1976680717 Introduce importlib.util.ModuleManager which is a context manager to Brett Cannon <brett@python.org> parents: 83791 diff changeset	523 sys.modules[self._name] = self._module
83998 39cc1b04713e Add a reset_name argument to importlib.util.module_to_load in order to Brett Cannon <brett@python.org> parents: 83988 diff changeset	524 elif self._reset_name:
39cc1b04713e Add a reset_name argument to importlib.util.module_to_load in order to Brett Cannon <brett@python.org> parents: 83988 diff changeset	525 try:
39cc1b04713e Add a reset_name argument to importlib.util.module_to_load in order to Brett Cannon <brett@python.org> parents: 83988 diff changeset	526 self._module.__name__ = self._name
39cc1b04713e Add a reset_name argument to importlib.util.module_to_load in order to Brett Cannon <brett@python.org> parents: 83988 diff changeset	527 except AttributeError:
39cc1b04713e Add a reset_name argument to importlib.util.module_to_load in order to Brett Cannon <brett@python.org> parents: 83988 diff changeset	528 pass
83960 3a1976680717 Introduce importlib.util.ModuleManager which is a context manager to Brett Cannon <brett@python.org> parents: 83791 diff changeset	529 return self._module
3a1976680717 Introduce importlib.util.ModuleManager which is a context manager to Brett Cannon <brett@python.org> parents: 83791 diff changeset	530
3a1976680717 Introduce importlib.util.ModuleManager which is a context manager to Brett Cannon <brett@python.org> parents: 83791 diff changeset	531 def __exit__(self, *args):
3a1976680717 Introduce importlib.util.ModuleManager which is a context manager to Brett Cannon <brett@python.org> parents: 83791 diff changeset	532 self._module.__initializing__ = False
3a1976680717 Introduce importlib.util.ModuleManager which is a context manager to Brett Cannon <brett@python.org> parents: 83791 diff changeset	533 del self._module
84171 205aa49e5cd5 Issue #18115: Abstract out managing the cleanup of modules to use in Brett Cannon <brett@python.org> parents: 84169 diff changeset	534 super().__exit__(*args)
83960 3a1976680717 Introduce importlib.util.ModuleManager which is a context manager to Brett Cannon <brett@python.org> parents: 83791 diff changeset	535
3a1976680717 Introduce importlib.util.ModuleManager which is a context manager to Brett Cannon <brett@python.org> parents: 83791 diff changeset	536
83999 130dfd6b3428 Fix for last commit on adding reset_name to module_to_load Brett Cannon <brett@python.org> parents: 83998 diff changeset	537 def module_to_load(name, *, reset_name=True):
130dfd6b3428 Fix for last commit on adding reset_name to module_to_load Brett Cannon <brett@python.org> parents: 83998 diff changeset	538 """Return a context manager which provides the module object to load.
130dfd6b3428 Fix for last commit on adding reset_name to module_to_load Brett Cannon <brett@python.org> parents: 83998 diff changeset	539
130dfd6b3428 Fix for last commit on adding reset_name to module_to_load Brett Cannon <brett@python.org> parents: 83998 diff changeset	540 If reset_name is true, reset the module's __name__ to 'name'.
130dfd6b3428 Fix for last commit on adding reset_name to module_to_load Brett Cannon <brett@python.org> parents: 83998 diff changeset	541 """
83988 6ace4133c1c0 Rename importlib.util.ModuleManager to module_to_load so that the name Brett Cannon <brett@python.org> parents: 83963 diff changeset	542 # Hiding _ModuleManager behind a function for better naming.
83999 130dfd6b3428 Fix for last commit on adding reset_name to module_to_load Brett Cannon <brett@python.org> parents: 83998 diff changeset	543 return _ModuleManager(name, reset_name=reset_name)
83988 6ace4133c1c0 Rename importlib.util.ModuleManager to module_to_load so that the name Brett Cannon <brett@python.org> parents: 83963 diff changeset	544
6ace4133c1c0 Rename importlib.util.ModuleManager to module_to_load so that the name Brett Cannon <brett@python.org> parents: 83963 diff changeset	545
84001 e873f2e67353 Issues #18088, 18089: Introduce Brett Cannon <brett@python.org> parents: 83999 diff changeset	546 def _init_package_attrs(loader, module):
e873f2e67353 Issues #18088, 18089: Introduce Brett Cannon <brett@python.org> parents: 83999 diff changeset	547 """Set __package__ and __path__ based on what loader.is_package() says."""
e873f2e67353 Issues #18088, 18089: Introduce Brett Cannon <brett@python.org> parents: 83999 diff changeset	548 name = module.__name__
e873f2e67353 Issues #18088, 18089: Introduce Brett Cannon <brett@python.org> parents: 83999 diff changeset	549 try:
e873f2e67353 Issues #18088, 18089: Introduce Brett Cannon <brett@python.org> parents: 83999 diff changeset	550 is_package = loader.is_package(name)
e873f2e67353 Issues #18088, 18089: Introduce Brett Cannon <brett@python.org> parents: 83999 diff changeset	551 except ImportError:
e873f2e67353 Issues #18088, 18089: Introduce Brett Cannon <brett@python.org> parents: 83999 diff changeset	552 pass
e873f2e67353 Issues #18088, 18089: Introduce Brett Cannon <brett@python.org> parents: 83999 diff changeset	553 else:
e873f2e67353 Issues #18088, 18089: Introduce Brett Cannon <brett@python.org> parents: 83999 diff changeset	554 if is_package:
e873f2e67353 Issues #18088, 18089: Introduce Brett Cannon <brett@python.org> parents: 83999 diff changeset	555 module.__package__ = name
e873f2e67353 Issues #18088, 18089: Introduce Brett Cannon <brett@python.org> parents: 83999 diff changeset	556 module.__path__ = []
e873f2e67353 Issues #18088, 18089: Introduce Brett Cannon <brett@python.org> parents: 83999 diff changeset	557 else:
e873f2e67353 Issues #18088, 18089: Introduce Brett Cannon <brett@python.org> parents: 83999 diff changeset	558 module.__package__ = name.rpartition('.')[0]
e873f2e67353 Issues #18088, 18089: Introduce Brett Cannon <brett@python.org> parents: 83999 diff changeset	559
e873f2e67353 Issues #18088, 18089: Introduce Brett Cannon <brett@python.org> parents: 83999 diff changeset	560
e873f2e67353 Issues #18088, 18089: Introduce Brett Cannon <brett@python.org> parents: 83999 diff changeset	561 def _init_file_attrs(loader, module):
e873f2e67353 Issues #18088, 18089: Introduce Brett Cannon <brett@python.org> parents: 83999 diff changeset	562 """Set __file__ and __path__ based on loader.get_filename()."""
e873f2e67353 Issues #18088, 18089: Introduce Brett Cannon <brett@python.org> parents: 83999 diff changeset	563 try:
e873f2e67353 Issues #18088, 18089: Introduce Brett Cannon <brett@python.org> parents: 83999 diff changeset	564 module.__file__ = loader.get_filename(module.__name__)
e873f2e67353 Issues #18088, 18089: Introduce Brett Cannon <brett@python.org> parents: 83999 diff changeset	565 except ImportError:
e873f2e67353 Issues #18088, 18089: Introduce Brett Cannon <brett@python.org> parents: 83999 diff changeset	566 pass
e873f2e67353 Issues #18088, 18089: Introduce Brett Cannon <brett@python.org> parents: 83999 diff changeset	567 else:
e873f2e67353 Issues #18088, 18089: Introduce Brett Cannon <brett@python.org> parents: 83999 diff changeset	568 if module.__name__ == module.__package__:
e873f2e67353 Issues #18088, 18089: Introduce Brett Cannon <brett@python.org> parents: 83999 diff changeset	569 module.__path__.append(_path_split(module.__file__)[0])
e873f2e67353 Issues #18088, 18089: Introduce Brett Cannon <brett@python.org> parents: 83999 diff changeset	570
e873f2e67353 Issues #18088, 18089: Introduce Brett Cannon <brett@python.org> parents: 83999 diff changeset	571
52002 76a579b6bade Rename importlib.util.set___package__ to set_package. Brett Cannon <bcannon@gmail.com> parents: 51995 diff changeset	572 def set_package(fxn):
51430 5cdc5f0c134c Move importlib completely over to using rpartition and accepting the empty Brett Cannon <bcannon@gmail.com> parents: 51429 diff changeset	573 """Set __package__ on the returned module."""
74449 9935f7837c4b Issue #13588: Rename decorators in importlib. Brett Cannon <brett@python.org> parents: 74358 diff changeset	574 def set_package_wrapper(args, *kwargs):
51430 5cdc5f0c134c Move importlib completely over to using rpartition and accepting the empty Brett Cannon <bcannon@gmail.com> parents: 51429 diff changeset	575 module = fxn(args, *kwargs)
77964 37850f083af5 Minor refactoring in importlib._bootstrap, and fix the '_wrap' docstring. Florent Xicluna <florent.xicluna@gmail.com> parents: 77917 diff changeset	576 if getattr(module, '__package__', None) is None:
51430 5cdc5f0c134c Move importlib completely over to using rpartition and accepting the empty Brett Cannon <bcannon@gmail.com> parents: 51429 diff changeset	577 module.__package__ = module.__name__
5cdc5f0c134c Move importlib completely over to using rpartition and accepting the empty Brett Cannon <bcannon@gmail.com> parents: 51429 diff changeset	578 if not hasattr(module, '__path__'):
5cdc5f0c134c Move importlib completely over to using rpartition and accepting the empty Brett Cannon <bcannon@gmail.com> parents: 51429 diff changeset	579 module.__package__ = module.__package__.rpartition('.')[0]
5cdc5f0c134c Move importlib completely over to using rpartition and accepting the empty Brett Cannon <bcannon@gmail.com> parents: 51429 diff changeset	580 return module
74449 9935f7837c4b Issue #13588: Rename decorators in importlib. Brett Cannon <brett@python.org> parents: 74358 diff changeset	581 _wrap(set_package_wrapper, fxn)
9935f7837c4b Issue #13588: Rename decorators in importlib. Brett Cannon <brett@python.org> parents: 74358 diff changeset	582 return set_package_wrapper
51430 5cdc5f0c134c Move importlib completely over to using rpartition and accepting the empty Brett Cannon <bcannon@gmail.com> parents: 51429 diff changeset	583
5cdc5f0c134c Move importlib completely over to using rpartition and accepting the empty Brett Cannon <bcannon@gmail.com> parents: 51429 diff changeset	584
52115 afd7118c8ccf Implement importlib.util.set_loader: a decorator to automatically set Brett Cannon <bcannon@gmail.com> parents: 52112 diff changeset	585 def set_loader(fxn):
afd7118c8ccf Implement importlib.util.set_loader: a decorator to automatically set Brett Cannon <bcannon@gmail.com> parents: 52112 diff changeset	586 """Set __loader__ on the returned module."""
74449 9935f7837c4b Issue #13588: Rename decorators in importlib. Brett Cannon <brett@python.org> parents: 74358 diff changeset	587 def set_loader_wrapper(self, args, *kwargs):
52115 afd7118c8ccf Implement importlib.util.set_loader: a decorator to automatically set Brett Cannon <bcannon@gmail.com> parents: 52112 diff changeset	588 module = fxn(self, args, *kwargs)
82643 7647aae9481c Issue #17117: Have both import itself and importlib.util.set_loader() Brett Cannon <brett@python.org> parents: 82385 diff changeset	589 if getattr(module, '__loader__', None) is None:
52115 afd7118c8ccf Implement importlib.util.set_loader: a decorator to automatically set Brett Cannon <bcannon@gmail.com> parents: 52112 diff changeset	590 module.__loader__ = self
afd7118c8ccf Implement importlib.util.set_loader: a decorator to automatically set Brett Cannon <bcannon@gmail.com> parents: 52112 diff changeset	591 return module
74449 9935f7837c4b Issue #13588: Rename decorators in importlib. Brett Cannon <brett@python.org> parents: 74358 diff changeset	592 _wrap(set_loader_wrapper, fxn)
9935f7837c4b Issue #13588: Rename decorators in importlib. Brett Cannon <brett@python.org> parents: 74358 diff changeset	593 return set_loader_wrapper
52115 afd7118c8ccf Implement importlib.util.set_loader: a decorator to automatically set Brett Cannon <bcannon@gmail.com> parents: 52112 diff changeset	594
afd7118c8ccf Implement importlib.util.set_loader: a decorator to automatically set Brett Cannon <bcannon@gmail.com> parents: 52112 diff changeset	595
52140 1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap. Brett Cannon <bcannon@gmail.com> parents: 52139 diff changeset	596 def _check_name(method):
1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap. Brett Cannon <bcannon@gmail.com> parents: 52139 diff changeset	597 """Decorator to verify that the module being requested matches the one the
1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap. Brett Cannon <bcannon@gmail.com> parents: 52139 diff changeset	598 loader can handle.
1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap. Brett Cannon <bcannon@gmail.com> parents: 52139 diff changeset	599
1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap. Brett Cannon <bcannon@gmail.com> parents: 52139 diff changeset	600 The first argument (self) must define _name which the second argument is
68488 0991b40e895d #11515: fix several typos. Patch by Piotr Kasprzyk. Ezio Melotti <ezio.melotti@gmail.com> parents: 58960 diff changeset	601 compared against. If the comparison fails then ImportError is raised.
52140 1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap. Brett Cannon <bcannon@gmail.com> parents: 52139 diff changeset	602
1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap. Brett Cannon <bcannon@gmail.com> parents: 52139 diff changeset	603 """
76877 626d5c6fbd95 Issue #13959: Have Brett Cannon <brett@python.org> parents: 76874 diff changeset	604 def _check_name_wrapper(self, name=None, args, *kwargs):
626d5c6fbd95 Issue #13959: Have Brett Cannon <brett@python.org> parents: 76874 diff changeset	605 if name is None:
626d5c6fbd95 Issue #13959: Have Brett Cannon <brett@python.org> parents: 76874 diff changeset	606 name = self.name
626d5c6fbd95 Issue #13959: Have Brett Cannon <brett@python.org> parents: 76874 diff changeset	607 elif self.name != name:
76280 c071af95772a Have importlib take advantage of ImportError's new 'name' and 'path' Brett Cannon <brett@python.org> parents: 76081 diff changeset	608 raise ImportError("loader cannot handle %s" % name, name=name)
52140 1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap. Brett Cannon <bcannon@gmail.com> parents: 52139 diff changeset	609 return method(self, name, args, *kwargs)
74449 9935f7837c4b Issue #13588: Rename decorators in importlib. Brett Cannon <brett@python.org> parents: 74358 diff changeset	610 _wrap(_check_name_wrapper, method)
9935f7837c4b Issue #13588: Rename decorators in importlib. Brett Cannon <brett@python.org> parents: 74358 diff changeset	611 return _check_name_wrapper
52140 1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap. Brett Cannon <bcannon@gmail.com> parents: 52139 diff changeset	612
1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap. Brett Cannon <bcannon@gmail.com> parents: 52139 diff changeset	613
52166 69cd615b76fb Implement InspectLoader for BuiltinImporter. Brett Cannon <bcannon@gmail.com> parents: 52165 diff changeset	614 def _requires_builtin(fxn):
69cd615b76fb Implement InspectLoader for BuiltinImporter. Brett Cannon <bcannon@gmail.com> parents: 52165 diff changeset	615 """Decorator to verify the named module is built-in."""
74449 9935f7837c4b Issue #13588: Rename decorators in importlib. Brett Cannon <brett@python.org> parents: 74358 diff changeset	616 def _requires_builtin_wrapper(self, fullname):
52166 69cd615b76fb Implement InspectLoader for BuiltinImporter. Brett Cannon <bcannon@gmail.com> parents: 52165 diff changeset	617 if fullname not in sys.builtin_module_names:
77964 37850f083af5 Minor refactoring in importlib._bootstrap, and fix the '_wrap' docstring. Florent Xicluna <florent.xicluna@gmail.com> parents: 77917 diff changeset	618 raise ImportError("{} is not a built-in module".format(fullname),
76280 c071af95772a Have importlib take advantage of ImportError's new 'name' and 'path' Brett Cannon <brett@python.org> parents: 76081 diff changeset	619 name=fullname)
52166 69cd615b76fb Implement InspectLoader for BuiltinImporter. Brett Cannon <bcannon@gmail.com> parents: 52165 diff changeset	620 return fxn(self, fullname)
74449 9935f7837c4b Issue #13588: Rename decorators in importlib. Brett Cannon <brett@python.org> parents: 74358 diff changeset	621 _wrap(_requires_builtin_wrapper, fxn)
9935f7837c4b Issue #13588: Rename decorators in importlib. Brett Cannon <brett@python.org> parents: 74358 diff changeset	622 return _requires_builtin_wrapper
52166 69cd615b76fb Implement InspectLoader for BuiltinImporter. Brett Cannon <bcannon@gmail.com> parents: 52165 diff changeset	623
69cd615b76fb Implement InspectLoader for BuiltinImporter. Brett Cannon <bcannon@gmail.com> parents: 52165 diff changeset	624
52168 70f5f9113ae1 Implement InspectLoader for FrozenImporter. Brett Cannon <bcannon@gmail.com> parents: 52166 diff changeset	625 def _requires_frozen(fxn):
70f5f9113ae1 Implement InspectLoader for FrozenImporter. Brett Cannon <bcannon@gmail.com> parents: 52166 diff changeset	626 """Decorator to verify the named module is frozen."""
74449 9935f7837c4b Issue #13588: Rename decorators in importlib. Brett Cannon <brett@python.org> parents: 74358 diff changeset	627 def _requires_frozen_wrapper(self, fullname):
76331 d777f854a66e Issue #13959: Rename imp to _imp and add Lib/imp.py and begin Brett Cannon <brett@python.org> parents: 76330 diff changeset	628 if not _imp.is_frozen(fullname):
77964 37850f083af5 Minor refactoring in importlib._bootstrap, and fix the '_wrap' docstring. Florent Xicluna <florent.xicluna@gmail.com> parents: 77917 diff changeset	629 raise ImportError("{} is not a frozen module".format(fullname),
76280 c071af95772a Have importlib take advantage of ImportError's new 'name' and 'path' Brett Cannon <brett@python.org> parents: 76081 diff changeset	630 name=fullname)
52168 70f5f9113ae1 Implement InspectLoader for FrozenImporter. Brett Cannon <bcannon@gmail.com> parents: 52166 diff changeset	631 return fxn(self, fullname)
74449 9935f7837c4b Issue #13588: Rename decorators in importlib. Brett Cannon <brett@python.org> parents: 74358 diff changeset	632 _wrap(_requires_frozen_wrapper, fxn)
9935f7837c4b Issue #13588: Rename decorators in importlib. Brett Cannon <brett@python.org> parents: 74358 diff changeset	633 return _requires_frozen_wrapper
52168 70f5f9113ae1 Implement InspectLoader for FrozenImporter. Brett Cannon <bcannon@gmail.com> parents: 52166 diff changeset	634
70f5f9113ae1 Implement InspectLoader for FrozenImporter. Brett Cannon <bcannon@gmail.com> parents: 52166 diff changeset	635
78489 e7a67f1bf604 Issue #15502: Refactor some code. Brett Cannon <brett@python.org> parents: 78487 diff changeset	636 def _find_module_shim(self, fullname):
e7a67f1bf604 Issue #15502: Refactor some code. Brett Cannon <brett@python.org> parents: 78487 diff changeset	637 """Try to find a loader for the specified module by delegating to
e7a67f1bf604 Issue #15502: Refactor some code. Brett Cannon <brett@python.org> parents: 78487 diff changeset	638 self.find_loader()."""
e7a67f1bf604 Issue #15502: Refactor some code. Brett Cannon <brett@python.org> parents: 78487 diff changeset	639 # Call find_loader(). If it returns a string (indicating this
e7a67f1bf604 Issue #15502: Refactor some code. Brett Cannon <brett@python.org> parents: 78487 diff changeset	640 # is a namespace package portion), generate a warning and
e7a67f1bf604 Issue #15502: Refactor some code. Brett Cannon <brett@python.org> parents: 78487 diff changeset	641 # return None.
e7a67f1bf604 Issue #15502: Refactor some code. Brett Cannon <brett@python.org> parents: 78487 diff changeset	642 loader, portions = self.find_loader(fullname)
e7a67f1bf604 Issue #15502: Refactor some code. Brett Cannon <brett@python.org> parents: 78487 diff changeset	643 if loader is None and len(portions):
e7a67f1bf604 Issue #15502: Refactor some code. Brett Cannon <brett@python.org> parents: 78487 diff changeset	644 msg = "Not importing directory {}: missing __init__"
e7a67f1bf604 Issue #15502: Refactor some code. Brett Cannon <brett@python.org> parents: 78487 diff changeset	645 _warnings.warn(msg.format(portions[0]), ImportWarning)
e7a67f1bf604 Issue #15502: Refactor some code. Brett Cannon <brett@python.org> parents: 78487 diff changeset	646 return loader
e7a67f1bf604 Issue #15502: Refactor some code. Brett Cannon <brett@python.org> parents: 78487 diff changeset	647
e7a67f1bf604 Issue #15502: Refactor some code. Brett Cannon <brett@python.org> parents: 78487 diff changeset	648
81432 a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	649 def _validate_bytecode_header(data, source_stats=None, name=None, path=None):
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	650 """Validate the header of the passed-in bytecode against source_stats (if
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	651 given) and returning the bytecode that can be compiled by compile().
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	652
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	653 All other arguments are used to enhance error reporting.
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	654
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	655 ImportError is raised when the magic number is incorrect or the bytecode is
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	656 found to be stale. EOFError is raised when the data is found to be
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	657 truncated.
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	658
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	659 """
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	660 exc_details = {}
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	661 if name is not None:
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	662 exc_details['name'] = name
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	663 else:
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	664 # To prevent having to make all messages have a conditional name.
81735 792810303239 Touch up exception messaging Brett Cannon <brett@python.org> parents: 81432 diff changeset	665 name = '<bytecode>'
81432 a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	666 if path is not None:
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	667 exc_details['path'] = path
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	668 magic = data[:4]
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	669 raw_timestamp = data[4:8]
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	670 raw_size = data[8:12]
84128 5619bc2d8207 Issue #18192: Introduce importlib.util.MAGIC_NUMBER and document the Brett Cannon <brett@python.org> parents: 84109 diff changeset	671 if magic != MAGIC_NUMBER:
83056 9a44e845a90e merge Brett Cannon <brett@python.org> parents: 82957 83055 diff changeset	672 message = 'bad magic number in {!r}: {!r}'.format(name, magic)
9a44e845a90e merge Brett Cannon <brett@python.org> parents: 82957 83055 diff changeset	673 _verbose_message(message)
9a44e845a90e merge Brett Cannon <brett@python.org> parents: 82957 83055 diff changeset	674 raise ImportError(message, **exc_details)
81432 a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	675 elif len(raw_timestamp) != 4:
84046 6b189f29b1e6 tweak exception message (again) Brett Cannon <brett@python.org> parents: 84023 diff changeset	676 message = 'reached EOF while reading timestamp in {!r}'.format(name)
81432 a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	677 _verbose_message(message)
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	678 raise EOFError(message)
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	679 elif len(raw_size) != 4:
84046 6b189f29b1e6 tweak exception message (again) Brett Cannon <brett@python.org> parents: 84023 diff changeset	680 message = 'reached EOF while reading size of source in {!r}'.format(name)
81432 a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	681 _verbose_message(message)
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	682 raise EOFError(message)
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	683 if source_stats is not None:
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	684 try:
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	685 source_mtime = int(source_stats['mtime'])
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	686 except KeyError:
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	687 pass
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	688 else:
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	689 if _r_long(raw_timestamp) != source_mtime:
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	690 message = 'bytecode is stale for {!r}'.format(name)
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	691 _verbose_message(message)
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	692 raise ImportError(message, **exc_details)
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	693 try:
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	694 source_size = source_stats['size'] & 0xFFFFFFFF
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	695 except KeyError:
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	696 pass
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	697 else:
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	698 if _r_long(raw_size) != source_size:
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	699 raise ImportError("bytecode is stale for {!r}".format(name),
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	700 **exc_details)
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	701 return data[12:]
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	702
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	703
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	704 def _compile_bytecode(data, name=None, bytecode_path=None, source_path=None):
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	705 """Compile bytecode as returned by _validate_bytecode_header()."""
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	706 code = marshal.loads(data)
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	707 if isinstance(code, _code_type):
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	708 _verbose_message('code object from {!r}', bytecode_path)
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	709 if source_path is not None:
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	710 _imp._fix_co_filename(code, source_path)
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	711 return code
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	712 else:
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	713 raise ImportError("Non-code object in {!r}".format(bytecode_path),
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	714 name=name, path=bytecode_path)
52141 f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	715
81761 d4eb02b6aac9 Port py_compile over to importlib Brett Cannon <brett@python.org> parents: 81735 diff changeset	716 def _code_to_bytecode(code, mtime=0, source_size=0):
d4eb02b6aac9 Port py_compile over to importlib Brett Cannon <brett@python.org> parents: 81735 diff changeset	717 """Compile a code object into bytecode for writing out to a byte-compiled
d4eb02b6aac9 Port py_compile over to importlib Brett Cannon <brett@python.org> parents: 81735 diff changeset	718 file."""
84128 5619bc2d8207 Issue #18192: Introduce importlib.util.MAGIC_NUMBER and document the Brett Cannon <brett@python.org> parents: 84109 diff changeset	719 data = bytearray(MAGIC_NUMBER)
81761 d4eb02b6aac9 Port py_compile over to importlib Brett Cannon <brett@python.org> parents: 81735 diff changeset	720 data.extend(_w_long(mtime))
d4eb02b6aac9 Port py_compile over to importlib Brett Cannon <brett@python.org> parents: 81735 diff changeset	721 data.extend(_w_long(source_size))
d4eb02b6aac9 Port py_compile over to importlib Brett Cannon <brett@python.org> parents: 81735 diff changeset	722 data.extend(marshal.dumps(code))
d4eb02b6aac9 Port py_compile over to importlib Brett Cannon <brett@python.org> parents: 81735 diff changeset	723 return data
52141 f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	724
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	725
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	726 # Loaders #####################################################################
52140 1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap. Brett Cannon <bcannon@gmail.com> parents: 52139 diff changeset	727
50983 0940a4f2506e Add importlib.machinery with its first tenants, BuitinImporter and Brett Cannon <bcannon@gmail.com> parents: 50893 diff changeset	728 class BuiltinImporter:
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	729
52165 62f9dad628de A few more docstring/API cleanups for importlib. Brett Cannon <bcannon@gmail.com> parents: 52164 diff changeset	730 """Meta path import for built-in modules.
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	731
52165 62f9dad628de A few more docstring/API cleanups for importlib. Brett Cannon <bcannon@gmail.com> parents: 52164 diff changeset	732 All methods are either class or static methods to avoid the need to
62f9dad628de A few more docstring/API cleanups for importlib. Brett Cannon <bcannon@gmail.com> parents: 52164 diff changeset	733 instantiate the class.
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	734
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	735 """
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	736
50983 0940a4f2506e Add importlib.machinery with its first tenants, BuitinImporter and Brett Cannon <bcannon@gmail.com> parents: 50893 diff changeset	737 @classmethod
77126 702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	738 def module_repr(cls, module):
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	739 return "<module '{}' (built-in)>".format(module.__name__)
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	740
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	741 @classmethod
50983 0940a4f2506e Add importlib.machinery with its first tenants, BuitinImporter and Brett Cannon <bcannon@gmail.com> parents: 50893 diff changeset	742 def find_module(cls, fullname, path=None):
52165 62f9dad628de A few more docstring/API cleanups for importlib. Brett Cannon <bcannon@gmail.com> parents: 52164 diff changeset	743 """Find the built-in module.
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	744
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	745 If 'path' is ever specified then the search is considered a failure.
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	746
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	747 """
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	748 if path is not None:
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	749 return None
76331 d777f854a66e Issue #13959: Rename imp to _imp and add Lib/imp.py and begin Brett Cannon <brett@python.org> parents: 76330 diff changeset	750 return cls if _imp.is_builtin(fullname) else None
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	751
51026 c26f1af26bfc Document both importlib.machinery.BuiltinImporter and FrozenImporter. Brett Cannon <bcannon@gmail.com> parents: 50983 diff changeset	752 @classmethod
52002 76a579b6bade Rename importlib.util.set___package__ to set_package. Brett Cannon <bcannon@gmail.com> parents: 51995 diff changeset	753 @set_package
52115 afd7118c8ccf Implement importlib.util.set_loader: a decorator to automatically set Brett Cannon <bcannon@gmail.com> parents: 52112 diff changeset	754 @set_loader
52166 69cd615b76fb Implement InspectLoader for BuiltinImporter. Brett Cannon <bcannon@gmail.com> parents: 52165 diff changeset	755 @_requires_builtin
51026 c26f1af26bfc Document both importlib.machinery.BuiltinImporter and FrozenImporter. Brett Cannon <bcannon@gmail.com> parents: 50983 diff changeset	756 def load_module(cls, fullname):
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	757 """Load a built-in module."""
84171 205aa49e5cd5 Issue #18115: Abstract out managing the cleanup of modules to use in Brett Cannon <brett@python.org> parents: 84169 diff changeset	758 with _ManageReload(fullname):
78344 62033490ca0f Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise Nick Coghlan <ncoghlan@gmail.com> parents: 78314 diff changeset	759 return _call_with_frames_removed(_imp.init_builtin, fullname)
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	760
52166 69cd615b76fb Implement InspectLoader for BuiltinImporter. Brett Cannon <bcannon@gmail.com> parents: 52165 diff changeset	761 @classmethod
69cd615b76fb Implement InspectLoader for BuiltinImporter. Brett Cannon <bcannon@gmail.com> parents: 52165 diff changeset	762 @_requires_builtin
69cd615b76fb Implement InspectLoader for BuiltinImporter. Brett Cannon <bcannon@gmail.com> parents: 52165 diff changeset	763 def get_code(cls, fullname):
69cd615b76fb Implement InspectLoader for BuiltinImporter. Brett Cannon <bcannon@gmail.com> parents: 52165 diff changeset	764 """Return None as built-in modules do not have code objects."""
69cd615b76fb Implement InspectLoader for BuiltinImporter. Brett Cannon <bcannon@gmail.com> parents: 52165 diff changeset	765 return None
69cd615b76fb Implement InspectLoader for BuiltinImporter. Brett Cannon <bcannon@gmail.com> parents: 52165 diff changeset	766
69cd615b76fb Implement InspectLoader for BuiltinImporter. Brett Cannon <bcannon@gmail.com> parents: 52165 diff changeset	767 @classmethod
69cd615b76fb Implement InspectLoader for BuiltinImporter. Brett Cannon <bcannon@gmail.com> parents: 52165 diff changeset	768 @_requires_builtin
69cd615b76fb Implement InspectLoader for BuiltinImporter. Brett Cannon <bcannon@gmail.com> parents: 52165 diff changeset	769 def get_source(cls, fullname):
69cd615b76fb Implement InspectLoader for BuiltinImporter. Brett Cannon <bcannon@gmail.com> parents: 52165 diff changeset	770 """Return None as built-in modules do not have source code."""
69cd615b76fb Implement InspectLoader for BuiltinImporter. Brett Cannon <bcannon@gmail.com> parents: 52165 diff changeset	771 return None
69cd615b76fb Implement InspectLoader for BuiltinImporter. Brett Cannon <bcannon@gmail.com> parents: 52165 diff changeset	772
69cd615b76fb Implement InspectLoader for BuiltinImporter. Brett Cannon <bcannon@gmail.com> parents: 52165 diff changeset	773 @classmethod
69cd615b76fb Implement InspectLoader for BuiltinImporter. Brett Cannon <bcannon@gmail.com> parents: 52165 diff changeset	774 @_requires_builtin
69cd615b76fb Implement InspectLoader for BuiltinImporter. Brett Cannon <bcannon@gmail.com> parents: 52165 diff changeset	775 def is_package(cls, fullname):
77964 37850f083af5 Minor refactoring in importlib._bootstrap, and fix the '_wrap' docstring. Florent Xicluna <florent.xicluna@gmail.com> parents: 77917 diff changeset	776 """Return False as built-in modules are never packages."""
52166 69cd615b76fb Implement InspectLoader for BuiltinImporter. Brett Cannon <bcannon@gmail.com> parents: 52165 diff changeset	777 return False
69cd615b76fb Implement InspectLoader for BuiltinImporter. Brett Cannon <bcannon@gmail.com> parents: 52165 diff changeset	778
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	779
50983 0940a4f2506e Add importlib.machinery with its first tenants, BuitinImporter and Brett Cannon <bcannon@gmail.com> parents: 50893 diff changeset	780 class FrozenImporter:
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	781
52165 62f9dad628de A few more docstring/API cleanups for importlib. Brett Cannon <bcannon@gmail.com> parents: 52164 diff changeset	782 """Meta path import for frozen modules.
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	783
52165 62f9dad628de A few more docstring/API cleanups for importlib. Brett Cannon <bcannon@gmail.com> parents: 52164 diff changeset	784 All methods are either class or static methods to avoid the need to
62f9dad628de A few more docstring/API cleanups for importlib. Brett Cannon <bcannon@gmail.com> parents: 52164 diff changeset	785 instantiate the class.
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	786
50983 0940a4f2506e Add importlib.machinery with its first tenants, BuitinImporter and Brett Cannon <bcannon@gmail.com> parents: 50893 diff changeset	787 """
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	788
50983 0940a4f2506e Add importlib.machinery with its first tenants, BuitinImporter and Brett Cannon <bcannon@gmail.com> parents: 50893 diff changeset	789 @classmethod
77126 702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	790 def module_repr(cls, m):
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	791 return "<module '{}' (frozen)>".format(m.__name__)
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	792
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	793 @classmethod
50983 0940a4f2506e Add importlib.machinery with its first tenants, BuitinImporter and Brett Cannon <bcannon@gmail.com> parents: 50893 diff changeset	794 def find_module(cls, fullname, path=None):
0940a4f2506e Add importlib.machinery with its first tenants, BuitinImporter and Brett Cannon <bcannon@gmail.com> parents: 50893 diff changeset	795 """Find a frozen module."""
76331 d777f854a66e Issue #13959: Rename imp to _imp and add Lib/imp.py and begin Brett Cannon <brett@python.org> parents: 76330 diff changeset	796 return cls if _imp.is_frozen(fullname) else None
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	797
50983 0940a4f2506e Add importlib.machinery with its first tenants, BuitinImporter and Brett Cannon <bcannon@gmail.com> parents: 50893 diff changeset	798 @classmethod
52002 76a579b6bade Rename importlib.util.set___package__ to set_package. Brett Cannon <bcannon@gmail.com> parents: 51995 diff changeset	799 @set_package
52115 afd7118c8ccf Implement importlib.util.set_loader: a decorator to automatically set Brett Cannon <bcannon@gmail.com> parents: 52112 diff changeset	800 @set_loader
52168 70f5f9113ae1 Implement InspectLoader for FrozenImporter. Brett Cannon <bcannon@gmail.com> parents: 52166 diff changeset	801 @_requires_frozen
50983 0940a4f2506e Add importlib.machinery with its first tenants, BuitinImporter and Brett Cannon <bcannon@gmail.com> parents: 50893 diff changeset	802 def load_module(cls, fullname):
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	803 """Load a frozen module."""
84171 205aa49e5cd5 Issue #18115: Abstract out managing the cleanup of modules to use in Brett Cannon <brett@python.org> parents: 84169 diff changeset	804 with _ManageReload(fullname):
78344 62033490ca0f Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise Nick Coghlan <ncoghlan@gmail.com> parents: 78314 diff changeset	805 m = _call_with_frames_removed(_imp.init_frozen, fullname)
77126 702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	806 # Let our own module_repr() method produce a suitable repr.
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	807 del m.__file__
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	808 return m
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	809
52168 70f5f9113ae1 Implement InspectLoader for FrozenImporter. Brett Cannon <bcannon@gmail.com> parents: 52166 diff changeset	810 @classmethod
70f5f9113ae1 Implement InspectLoader for FrozenImporter. Brett Cannon <bcannon@gmail.com> parents: 52166 diff changeset	811 @_requires_frozen
70f5f9113ae1 Implement InspectLoader for FrozenImporter. Brett Cannon <bcannon@gmail.com> parents: 52166 diff changeset	812 def get_code(cls, fullname):
70f5f9113ae1 Implement InspectLoader for FrozenImporter. Brett Cannon <bcannon@gmail.com> parents: 52166 diff changeset	813 """Return the code object for the frozen module."""
76331 d777f854a66e Issue #13959: Rename imp to _imp and add Lib/imp.py and begin Brett Cannon <brett@python.org> parents: 76330 diff changeset	814 return _imp.get_frozen_object(fullname)
52168 70f5f9113ae1 Implement InspectLoader for FrozenImporter. Brett Cannon <bcannon@gmail.com> parents: 52166 diff changeset	815
70f5f9113ae1 Implement InspectLoader for FrozenImporter. Brett Cannon <bcannon@gmail.com> parents: 52166 diff changeset	816 @classmethod
70f5f9113ae1 Implement InspectLoader for FrozenImporter. Brett Cannon <bcannon@gmail.com> parents: 52166 diff changeset	817 @_requires_frozen
70f5f9113ae1 Implement InspectLoader for FrozenImporter. Brett Cannon <bcannon@gmail.com> parents: 52166 diff changeset	818 def get_source(cls, fullname):
70f5f9113ae1 Implement InspectLoader for FrozenImporter. Brett Cannon <bcannon@gmail.com> parents: 52166 diff changeset	819 """Return None as frozen modules do not have source code."""
70f5f9113ae1 Implement InspectLoader for FrozenImporter. Brett Cannon <bcannon@gmail.com> parents: 52166 diff changeset	820 return None
70f5f9113ae1 Implement InspectLoader for FrozenImporter. Brett Cannon <bcannon@gmail.com> parents: 52166 diff changeset	821
70f5f9113ae1 Implement InspectLoader for FrozenImporter. Brett Cannon <bcannon@gmail.com> parents: 52166 diff changeset	822 @classmethod
70f5f9113ae1 Implement InspectLoader for FrozenImporter. Brett Cannon <bcannon@gmail.com> parents: 52166 diff changeset	823 @_requires_frozen
70f5f9113ae1 Implement InspectLoader for FrozenImporter. Brett Cannon <bcannon@gmail.com> parents: 52166 diff changeset	824 def is_package(cls, fullname):
78491 f7b59e890e30 fix docstring wording Philip Jenvey <pjenvey@underboss.org> parents: 78489 diff changeset	825 """Return True if the frozen module is a package."""
76331 d777f854a66e Issue #13959: Rename imp to _imp and add Lib/imp.py and begin Brett Cannon <brett@python.org> parents: 76330 diff changeset	826 return _imp.is_frozen_package(fullname)
52168 70f5f9113ae1 Implement InspectLoader for FrozenImporter. Brett Cannon <bcannon@gmail.com> parents: 52166 diff changeset	827
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	828
78380 a1ac1e13c5a0 Close #15519: Properly expose WindowsRegistryFinder in importlib and bring the name into line with normal import terminology. Original patch by Eric Snow Nick Coghlan <ncoghlan@gmail.com> parents: 78344 diff changeset	829 class WindowsRegistryFinder:
78308 bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	830
78380 a1ac1e13c5a0 Close #15519: Properly expose WindowsRegistryFinder in importlib and bring the name into line with normal import terminology. Original patch by Eric Snow Nick Coghlan <ncoghlan@gmail.com> parents: 78344 diff changeset	831 """Meta path finder for modules declared in the Windows registry.
78308 bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	832 """
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	833
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	834 REGISTRY_KEY = (
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	835 "Software\\Python\\PythonCore\\{sys_version}"
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	836 "\\Modules\\{fullname}")
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	837 REGISTRY_KEY_DEBUG = (
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	838 "Software\\Python\\PythonCore\\{sys_version}"
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	839 "\\Modules\\{fullname}\\Debug")
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	840 DEBUG_BUILD = False # Changed in _setup()
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	841
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	842 @classmethod
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	843 def _open_registry(cls, key):
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	844 try:
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	845 return _winreg.OpenKey(_winreg.HKEY_CURRENT_USER, key)
80944 bb94f6222fef Issue #16719: Get rid of WindowsError. Use OSError instead Andrew Svetlov <andrew.svetlov@gmail.com> parents: 80930 diff changeset	846 except OSError:
78308 bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	847 return _winreg.OpenKey(_winreg.HKEY_LOCAL_MACHINE, key)
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	848
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	849 @classmethod
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	850 def _search_registry(cls, fullname):
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	851 if cls.DEBUG_BUILD:
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	852 registry_key = cls.REGISTRY_KEY_DEBUG
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	853 else:
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	854 registry_key = cls.REGISTRY_KEY
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	855 key = registry_key.format(fullname=fullname,
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	856 sys_version=sys.version[:3])
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	857 try:
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	858 with cls._open_registry(key) as hkey:
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	859 filepath = _winreg.QueryValue(hkey, "")
80944 bb94f6222fef Issue #16719: Get rid of WindowsError. Use OSError instead Andrew Svetlov <andrew.svetlov@gmail.com> parents: 80930 diff changeset	860 except OSError:
78308 bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	861 return None
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	862 return filepath
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	863
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	864 @classmethod
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	865 def find_module(cls, fullname, path=None):
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	866 """Find module named in the registry."""
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	867 filepath = cls._search_registry(fullname)
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	868 if filepath is None:
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	869 return None
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	870 try:
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	871 _os.stat(filepath)
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	872 except OSError:
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	873 return None
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	874 for loader, suffixes, _ in _get_supported_file_loaders():
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	875 if filepath.endswith(tuple(suffixes)):
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	876 return loader(fullname, filepath)
52168 70f5f9113ae1 Implement InspectLoader for FrozenImporter. Brett Cannon <bcannon@gmail.com> parents: 52166 diff changeset	877
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	878
62756 34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	879 class _LoaderBasics:
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	880
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	881 """Base class of common code needed by both SourceLoader and
76478 1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and Brett Cannon <brett@python.org> parents: 76472 diff changeset	882 SourcelessFileLoader."""
51771 ee7029710c05 Refactor source and bytecode file loaders in importlib so that there Brett Cannon <bcannon@gmail.com> parents: 51770 diff changeset	883
62756 34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	884 def is_package(self, fullname):
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	885 """Concrete implementation of InspectLoader.is_package by checking if
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	886 the path returned by get_filename has a filename of '__init__.py'."""
76449 163360370ad2 Make path manipulation more robust for platforms with alternative path Brett Cannon <brett@python.org> parents: 76445 diff changeset	887 filename = _path_split(self.get_filename(fullname))[1]
77457 240b7467e65c Issue #14938: importlib.abc.SourceLoader.is_package() now takes the Brett Cannon <brett@python.org> parents: 77180 diff changeset	888 filename_base = filename.rsplit('.', 1)[0]
240b7467e65c Issue #14938: importlib.abc.SourceLoader.is_package() now takes the Brett Cannon <brett@python.org> parents: 77180 diff changeset	889 tail_name = fullname.rpartition('.')[2]
240b7467e65c Issue #14938: importlib.abc.SourceLoader.is_package() now takes the Brett Cannon <brett@python.org> parents: 77180 diff changeset	890 return filename_base == '__init__' and tail_name != '__init__'
52165 62f9dad628de A few more docstring/API cleanups for importlib. Brett Cannon <bcannon@gmail.com> parents: 52164 diff changeset	891
84001 e873f2e67353 Issues #18088, 18089: Introduce Brett Cannon <brett@python.org> parents: 83999 diff changeset	892 def init_module_attrs(self, module):
e873f2e67353 Issues #18088, 18089: Introduce Brett Cannon <brett@python.org> parents: 83999 diff changeset	893 """Set various attributes on the module.
e873f2e67353 Issues #18088, 18089: Introduce Brett Cannon <brett@python.org> parents: 83999 diff changeset	894
e873f2e67353 Issues #18088, 18089: Introduce Brett Cannon <brett@python.org> parents: 83999 diff changeset	895 ExecutionLoader.init_module_attrs() is used to set __loader__,
e873f2e67353 Issues #18088, 18089: Introduce Brett Cannon <brett@python.org> parents: 83999 diff changeset	896 __package__, __file__, and optionally __path__. The __cached__ attribute
e873f2e67353 Issues #18088, 18089: Introduce Brett Cannon <brett@python.org> parents: 83999 diff changeset	897 is set using imp.cache_from_source() and __file__.
e873f2e67353 Issues #18088, 18089: Introduce Brett Cannon <brett@python.org> parents: 83999 diff changeset	898 """
e873f2e67353 Issues #18088, 18089: Introduce Brett Cannon <brett@python.org> parents: 83999 diff changeset	899 module.__loader__ = self # Loader
e873f2e67353 Issues #18088, 18089: Introduce Brett Cannon <brett@python.org> parents: 83999 diff changeset	900 _init_package_attrs(self, module) # InspectLoader
e873f2e67353 Issues #18088, 18089: Introduce Brett Cannon <brett@python.org> parents: 83999 diff changeset	901 _init_file_attrs(self, module) # ExecutionLoader
e873f2e67353 Issues #18088, 18089: Introduce Brett Cannon <brett@python.org> parents: 83999 diff changeset	902 if hasattr(module, '__file__'): # SourceLoader
78026 e86330669bb5 Issue #15056: imp.cache_from_source() and source_from_cache() raise Brett Cannon <brett@python.org> parents: 78010 diff changeset	903 try:
e86330669bb5 Issue #15056: imp.cache_from_source() and source_from_cache() raise Brett Cannon <brett@python.org> parents: 78010 diff changeset	904 module.__cached__ = cache_from_source(module.__file__)
e86330669bb5 Issue #15056: imp.cache_from_source() and source_from_cache() raise Brett Cannon <brett@python.org> parents: 78010 diff changeset	905 except NotImplementedError:
84001 e873f2e67353 Issues #18088, 18089: Introduce Brett Cannon <brett@python.org> parents: 83999 diff changeset	906 pass
e873f2e67353 Issues #18088, 18089: Introduce Brett Cannon <brett@python.org> parents: 83999 diff changeset	907
e873f2e67353 Issues #18088, 18089: Introduce Brett Cannon <brett@python.org> parents: 83999 diff changeset	908 def load_module(self, fullname):
e873f2e67353 Issues #18088, 18089: Introduce Brett Cannon <brett@python.org> parents: 83999 diff changeset	909 """Load the specified module into sys.modules and return it."""
e873f2e67353 Issues #18088, 18089: Introduce Brett Cannon <brett@python.org> parents: 83999 diff changeset	910 with module_to_load(fullname) as module:
e873f2e67353 Issues #18088, 18089: Introduce Brett Cannon <brett@python.org> parents: 83999 diff changeset	911 self.init_module_attrs(module)
e873f2e67353 Issues #18088, 18089: Introduce Brett Cannon <brett@python.org> parents: 83999 diff changeset	912 code = self.get_code(fullname)
e873f2e67353 Issues #18088, 18089: Introduce Brett Cannon <brett@python.org> parents: 83999 diff changeset	913 if code is None:
e873f2e67353 Issues #18088, 18089: Introduce Brett Cannon <brett@python.org> parents: 83999 diff changeset	914 raise ImportError('cannot load module {!r} when get_code() '
e873f2e67353 Issues #18088, 18089: Introduce Brett Cannon <brett@python.org> parents: 83999 diff changeset	915 'returns None'.format(fullname))
e873f2e67353 Issues #18088, 18089: Introduce Brett Cannon <brett@python.org> parents: 83999 diff changeset	916 _call_with_frames_removed(exec, code, module.__dict__)
e873f2e67353 Issues #18088, 18089: Introduce Brett Cannon <brett@python.org> parents: 83999 diff changeset	917 return module
51771 ee7029710c05 Refactor source and bytecode file loaders in importlib so that there Brett Cannon <bcannon@gmail.com> parents: 51770 diff changeset	918
62756 34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	919
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	920 class SourceLoader(_LoaderBasics):
51771 ee7029710c05 Refactor source and bytecode file loaders in importlib so that there Brett Cannon <bcannon@gmail.com> parents: 51770 diff changeset	921
67573 e125d92f7438 Issue 10899: Remove function type annotations from the stdlib Raymond Hettinger <python@rcn.com> parents: 66226 diff changeset	922 def path_mtime(self, path):
67581 7d658b49b3e1 Issue 10899: Remove function type annotations from the stdlib Raymond Hettinger <python@rcn.com> parents: 67573 diff changeset	923 """Optional method that returns the modification time (an int) for the
7d658b49b3e1 Issue 10899: Remove function type annotations from the stdlib Raymond Hettinger <python@rcn.com> parents: 67573 diff changeset	924 specified path, where path is a str.
83225 8e733e30edf6 Issue #17093,17566,17567: Methods from classes in importlib.abc now raise/return Brett Cannon <brett@python.org> parents: 83056 diff changeset	925
8e733e30edf6 Issue #17093,17566,17567: Methods from classes in importlib.abc now raise/return Brett Cannon <brett@python.org> parents: 83056 diff changeset	926 Raises IOError when the path cannot be handled.
74358 87331661042b Issue #13645: pyc files now contain the size of the corresponding source Antoine Pitrou <solipsis@pitrou.net> parents: 73977 diff changeset	927 """
83225 8e733e30edf6 Issue #17093,17566,17567: Methods from classes in importlib.abc now raise/return Brett Cannon <brett@python.org> parents: 83056 diff changeset	928 raise IOError
74358 87331661042b Issue #13645: pyc files now contain the size of the corresponding source Antoine Pitrou <solipsis@pitrou.net> parents: 73977 diff changeset	929
87331661042b Issue #13645: pyc files now contain the size of the corresponding source Antoine Pitrou <solipsis@pitrou.net> parents: 73977 diff changeset	930 def path_stats(self, path):
87331661042b Issue #13645: pyc files now contain the size of the corresponding source Antoine Pitrou <solipsis@pitrou.net> parents: 73977 diff changeset	931 """Optional method returning a metadata dict for the specified path
87331661042b Issue #13645: pyc files now contain the size of the corresponding source Antoine Pitrou <solipsis@pitrou.net> parents: 73977 diff changeset	932 to by the path (str).
87331661042b Issue #13645: pyc files now contain the size of the corresponding source Antoine Pitrou <solipsis@pitrou.net> parents: 73977 diff changeset	933 Possible keys:
87331661042b Issue #13645: pyc files now contain the size of the corresponding source Antoine Pitrou <solipsis@pitrou.net> parents: 73977 diff changeset	934 - 'mtime' (mandatory) is the numeric timestamp of last source
87331661042b Issue #13645: pyc files now contain the size of the corresponding source Antoine Pitrou <solipsis@pitrou.net> parents: 73977 diff changeset	935 code modification;
87331661042b Issue #13645: pyc files now contain the size of the corresponding source Antoine Pitrou <solipsis@pitrou.net> parents: 73977 diff changeset	936 - 'size' (optional) is the size in bytes of the source code.
52112 03080fa08dfe Implement get_source for importlib.abc.PyLoader using source_path and get_data. Brett Cannon <bcannon@gmail.com> parents: 52083 diff changeset	937
62587 b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	938 Implementing this method allows the loader to read bytecode files.
83225 8e733e30edf6 Issue #17093,17566,17567: Methods from classes in importlib.abc now raise/return Brett Cannon <brett@python.org> parents: 83056 diff changeset	939 Raises IOError when the path cannot be handled.
52112 03080fa08dfe Implement get_source for importlib.abc.PyLoader using source_path and get_data. Brett Cannon <bcannon@gmail.com> parents: 52083 diff changeset	940 """
74358 87331661042b Issue #13645: pyc files now contain the size of the corresponding source Antoine Pitrou <solipsis@pitrou.net> parents: 73977 diff changeset	941 return {'mtime': self.path_mtime(path)}
62587 b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	942
78722 3a831a0a29c4 Close #2501: Permission bits are once again correctly copied from the source file to the cached bytecode file. Test by Eric Snow. Nick Coghlan <ncoghlan@gmail.com> parents: 78663 diff changeset	943 def _cache_bytecode(self, source_path, cache_path, data):
3a831a0a29c4 Close #2501: Permission bits are once again correctly copied from the source file to the cached bytecode file. Test by Eric Snow. Nick Coghlan <ncoghlan@gmail.com> parents: 78663 diff changeset	944 """Optional method which writes data (bytes) to a file path (a str).
3a831a0a29c4 Close #2501: Permission bits are once again correctly copied from the source file to the cached bytecode file. Test by Eric Snow. Nick Coghlan <ncoghlan@gmail.com> parents: 78663 diff changeset	945
3a831a0a29c4 Close #2501: Permission bits are once again correctly copied from the source file to the cached bytecode file. Test by Eric Snow. Nick Coghlan <ncoghlan@gmail.com> parents: 78663 diff changeset	946 Implementing this method allows for the writing of bytecode files.
3a831a0a29c4 Close #2501: Permission bits are once again correctly copied from the source file to the cached bytecode file. Test by Eric Snow. Nick Coghlan <ncoghlan@gmail.com> parents: 78663 diff changeset	947
3a831a0a29c4 Close #2501: Permission bits are once again correctly copied from the source file to the cached bytecode file. Test by Eric Snow. Nick Coghlan <ncoghlan@gmail.com> parents: 78663 diff changeset	948 The source path is needed in order to correctly transfer permissions
3a831a0a29c4 Close #2501: Permission bits are once again correctly copied from the source file to the cached bytecode file. Test by Eric Snow. Nick Coghlan <ncoghlan@gmail.com> parents: 78663 diff changeset	949 """
3a831a0a29c4 Close #2501: Permission bits are once again correctly copied from the source file to the cached bytecode file. Test by Eric Snow. Nick Coghlan <ncoghlan@gmail.com> parents: 78663 diff changeset	950 # For backwards compatibility, we delegate to set_data()
3a831a0a29c4 Close #2501: Permission bits are once again correctly copied from the source file to the cached bytecode file. Test by Eric Snow. Nick Coghlan <ncoghlan@gmail.com> parents: 78663 diff changeset	951 return self.set_data(cache_path, data)
62587 b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	952
67573 e125d92f7438 Issue 10899: Remove function type annotations from the stdlib Raymond Hettinger <python@rcn.com> parents: 66226 diff changeset	953 def set_data(self, path, data):
67581 7d658b49b3e1 Issue 10899: Remove function type annotations from the stdlib Raymond Hettinger <python@rcn.com> parents: 67573 diff changeset	954 """Optional method which writes data (bytes) to a file path (a str).
62587 b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	955
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	956 Implementing this method allows for the writing of bytecode files.
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	957 """
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	958
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	959
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	960 def get_source(self, fullname):
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	961 """Concrete implementation of InspectLoader.get_source."""
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	962 path = self.get_filename(fullname)
52112 03080fa08dfe Implement get_source for importlib.abc.PyLoader using source_path and get_data. Brett Cannon <bcannon@gmail.com> parents: 52083 diff changeset	963 try:
03080fa08dfe Implement get_source for importlib.abc.PyLoader using source_path and get_data. Brett Cannon <bcannon@gmail.com> parents: 52083 diff changeset	964 source_bytes = self.get_data(path)
81039 7d69d04522e3 Replace IOError with OSError (#16715) Andrew Svetlov <andrew.svetlov@gmail.com> parents: 80944 diff changeset	965 except OSError as exc:
76280 c071af95772a Have importlib take advantage of ImportError's new 'name' and 'path' Brett Cannon <brett@python.org> parents: 76081 diff changeset	966 raise ImportError("source not available through get_data()",
78112 7d202353a728 Issue #15343: A lot more than just unicode decoding can go wrong when retrieving a source file Nick Coghlan <ncoghlan@gmail.com> parents: 78078 diff changeset	967 name=fullname) from exc
84172 e353f64dfd95 importlib.abc.SourceLoader.get_source() was re-raising SyntaxError and Brett Cannon <brett@python.org> parents: 84171 diff changeset	968 import tokenize
78112 7d202353a728 Issue #15343: A lot more than just unicode decoding can go wrong when retrieving a source file Nick Coghlan <ncoghlan@gmail.com> parents: 78078 diff changeset	969 readsource = _io.BytesIO(source_bytes).readline
84172 e353f64dfd95 importlib.abc.SourceLoader.get_source() was re-raising SyntaxError and Brett Cannon <brett@python.org> parents: 84171 diff changeset	970 encoding = tokenize.detect_encoding(readsource)
62761 32682b253ffc Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62756 diff changeset	971 newline_decoder = _io.IncrementalNewlineDecoder(None, True)
84172 e353f64dfd95 importlib.abc.SourceLoader.get_source() was re-raising SyntaxError and Brett Cannon <brett@python.org> parents: 84171 diff changeset	972 return newline_decoder.decode(source_bytes.decode(encoding[0]))
51771 ee7029710c05 Refactor source and bytecode file loaders in importlib so that there Brett Cannon <bcannon@gmail.com> parents: 51770 diff changeset	973
81761 d4eb02b6aac9 Port py_compile over to importlib Brett Cannon <brett@python.org> parents: 81735 diff changeset	974 def source_to_code(self, data, path, *, _optimize=-1):
80507 e30bcce5c634 Issue #15627: Add the compile_source() method to Brett Cannon <brett@python.org> parents: 80410 diff changeset	975 """Return the code object compiled from source.
e30bcce5c634 Issue #15627: Add the compile_source() method to Brett Cannon <brett@python.org> parents: 80410 diff changeset	976
e30bcce5c634 Issue #15627: Add the compile_source() method to Brett Cannon <brett@python.org> parents: 80410 diff changeset	977 The 'data' argument can be any object type that compile() supports.
e30bcce5c634 Issue #15627: Add the compile_source() method to Brett Cannon <brett@python.org> parents: 80410 diff changeset	978 """
e30bcce5c634 Issue #15627: Add the compile_source() method to Brett Cannon <brett@python.org> parents: 80410 diff changeset	979 return _call_with_frames_removed(compile, data, path, 'exec',
81761 d4eb02b6aac9 Port py_compile over to importlib Brett Cannon <brett@python.org> parents: 81735 diff changeset	980 dont_inherit=True, optimize=_optimize)
80507 e30bcce5c634 Issue #15627: Add the compile_source() method to Brett Cannon <brett@python.org> parents: 80410 diff changeset	981
51771 ee7029710c05 Refactor source and bytecode file loaders in importlib so that there Brett Cannon <bcannon@gmail.com> parents: 51770 diff changeset	982 def get_code(self, fullname):
62587 b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	983 """Concrete implementation of InspectLoader.get_code.
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	984
74358 87331661042b Issue #13645: pyc files now contain the size of the corresponding source Antoine Pitrou <solipsis@pitrou.net> parents: 73977 diff changeset	985 Reading of bytecode requires path_stats to be implemented. To write
62587 b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	986 bytecode, set_data must also be implemented.
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	987
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	988 """
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	989 source_path = self.get_filename(fullname)
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	990 source_mtime = None
78026 e86330669bb5 Issue #15056: imp.cache_from_source() and source_from_cache() raise Brett Cannon <brett@python.org> parents: 78010 diff changeset	991 try:
e86330669bb5 Issue #15056: imp.cache_from_source() and source_from_cache() raise Brett Cannon <brett@python.org> parents: 78010 diff changeset	992 bytecode_path = cache_from_source(source_path)
e86330669bb5 Issue #15056: imp.cache_from_source() and source_from_cache() raise Brett Cannon <brett@python.org> parents: 78010 diff changeset	993 except NotImplementedError:
e86330669bb5 Issue #15056: imp.cache_from_source() and source_from_cache() raise Brett Cannon <brett@python.org> parents: 78010 diff changeset	994 bytecode_path = None
e86330669bb5 Issue #15056: imp.cache_from_source() and source_from_cache() raise Brett Cannon <brett@python.org> parents: 78010 diff changeset	995 else:
51771 ee7029710c05 Refactor source and bytecode file loaders in importlib so that there Brett Cannon <bcannon@gmail.com> parents: 51770 diff changeset	996 try:
74358 87331661042b Issue #13645: pyc files now contain the size of the corresponding source Antoine Pitrou <solipsis@pitrou.net> parents: 73977 diff changeset	997 st = self.path_stats(source_path)
83225 8e733e30edf6 Issue #17093,17566,17567: Methods from classes in importlib.abc now raise/return Brett Cannon <brett@python.org> parents: 83056 diff changeset	998 except IOError:
62587 b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	999 pass
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	1000 else:
74358 87331661042b Issue #13645: pyc files now contain the size of the corresponding source Antoine Pitrou <solipsis@pitrou.net> parents: 73977 diff changeset	1001 source_mtime = int(st['mtime'])
62587 b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	1002 try:
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	1003 data = self.get_data(bytecode_path)
81039 7d69d04522e3 Replace IOError with OSError (#16715) Andrew Svetlov <andrew.svetlov@gmail.com> parents: 80944 diff changeset	1004 except OSError:
51771 ee7029710c05 Refactor source and bytecode file loaders in importlib so that there Brett Cannon <bcannon@gmail.com> parents: 51770 diff changeset	1005 pass
ee7029710c05 Refactor source and bytecode file loaders in importlib so that there Brett Cannon <bcannon@gmail.com> parents: 51770 diff changeset	1006 else:
62756 34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	1007 try:
81432 a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	1008 bytes_data = _validate_bytecode_header(data,
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	1009 source_stats=st, name=fullname,
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	1010 path=bytecode_path)
62756 34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	1011 except (ImportError, EOFError):
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	1012 pass
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	1013 else:
76753 59b8a7fcb047 Properly mark names in importlib._bootstrap as private. Brett Cannon <brett@python.org> parents: 76586 diff changeset	1014 _verbose_message('{} matches {}', bytecode_path,
76303 2dd046be2c88 Issue #2377: Make importlib the implementation of __import__(). Brett Cannon <brett@python.org> parents: 76280 diff changeset	1015 source_path)
81432 a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	1016 return _compile_bytecode(bytes_data, name=fullname,
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	1017 bytecode_path=bytecode_path,
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	1018 source_path=source_path)
62587 b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	1019 source_bytes = self.get_data(source_path)
80725 7caab48f171e Issue #15627: This is simply an update to the name of a new method recently added Eric Snow <ericsnowcurrently@gmail.com> parents: 80530 diff changeset	1020 code_object = self.source_to_code(source_bytes, source_path)
76753 59b8a7fcb047 Properly mark names in importlib._bootstrap as private. Brett Cannon <brett@python.org> parents: 76586 diff changeset	1021 _verbose_message('code object from {}', source_path)
62587 b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	1022 if (not sys.dont_write_bytecode and bytecode_path is not None and
81761 d4eb02b6aac9 Port py_compile over to importlib Brett Cannon <brett@python.org> parents: 81735 diff changeset	1023 source_mtime is not None):
d4eb02b6aac9 Port py_compile over to importlib Brett Cannon <brett@python.org> parents: 81735 diff changeset	1024 data = _code_to_bytecode(code_object, source_mtime,
d4eb02b6aac9 Port py_compile over to importlib Brett Cannon <brett@python.org> parents: 81735 diff changeset	1025 len(source_bytes))
62587 b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	1026 try:
78722 3a831a0a29c4 Close #2501: Permission bits are once again correctly copied from the source file to the cached bytecode file. Test by Eric Snow. Nick Coghlan <ncoghlan@gmail.com> parents: 78663 diff changeset	1027 self._cache_bytecode(source_path, bytecode_path, data)
76753 59b8a7fcb047 Properly mark names in importlib._bootstrap as private. Brett Cannon <brett@python.org> parents: 76586 diff changeset	1028 _verbose_message('wrote {!r}', bytecode_path)
62587 b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	1029 except NotImplementedError:
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	1030 pass
51771 ee7029710c05 Refactor source and bytecode file loaders in importlib so that there Brett Cannon <bcannon@gmail.com> parents: 51770 diff changeset	1031 return code_object
ee7029710c05 Refactor source and bytecode file loaders in importlib so that there Brett Cannon <bcannon@gmail.com> parents: 51770 diff changeset	1032
ee7029710c05 Refactor source and bytecode file loaders in importlib so that there Brett Cannon <bcannon@gmail.com> parents: 51770 diff changeset	1033
76478 1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and Brett Cannon <brett@python.org> parents: 76472 diff changeset	1034 class FileLoader:
51771 ee7029710c05 Refactor source and bytecode file loaders in importlib so that there Brett Cannon <bcannon@gmail.com> parents: 51770 diff changeset	1035
62756 34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	1036 """Base file loader class which implements the loader protocol methods that
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	1037 require file system usage."""
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	1038
62587 b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	1039 def __init__(self, fullname, path):
62756 34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	1040 """Cache the module name and the path to the file found by the
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	1041 finder."""
76478 1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and Brett Cannon <brett@python.org> parents: 76472 diff changeset	1042 self.name = fullname
1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and Brett Cannon <brett@python.org> parents: 76472 diff changeset	1043 self.path = path
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	1044
52139 2e8e897a1244 Make utility code in importlib._bootstrap private. Brett Cannon <bcannon@gmail.com> parents: 52115 diff changeset	1045 @_check_name
76877 626d5c6fbd95 Issue #13959: Have Brett Cannon <brett@python.org> parents: 76874 diff changeset	1046 def load_module(self, fullname):
626d5c6fbd95 Issue #13959: Have Brett Cannon <brett@python.org> parents: 76874 diff changeset	1047 """Load a module from a file."""
77180 bcb3b81853cc Tweak importlib._bootstrap to avoid zero-argument super so I can work on issue #14857 without breaking imports Nick Coghlan <ncoghlan@gmail.com> parents: 77164 diff changeset	1048 # Issue #14857: Avoid the zero-argument form so the implementation
bcb3b81853cc Tweak importlib._bootstrap to avoid zero-argument super so I can work on issue #14857 without breaking imports Nick Coghlan <ncoghlan@gmail.com> parents: 77164 diff changeset	1049 # of that form can be updated without breaking the frozen module
bcb3b81853cc Tweak importlib._bootstrap to avoid zero-argument super so I can work on issue #14857 without breaking imports Nick Coghlan <ncoghlan@gmail.com> parents: 77164 diff changeset	1050 return super(FileLoader, self).load_module(fullname)
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	1051
52139 2e8e897a1244 Make utility code in importlib._bootstrap private. Brett Cannon <bcannon@gmail.com> parents: 52115 diff changeset	1052 @_check_name
62587 b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	1053 def get_filename(self, fullname):
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	1054 """Return the path to the source file as found by the finder."""
76478 1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and Brett Cannon <brett@python.org> parents: 76472 diff changeset	1055 return self.path
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	1056
51771 ee7029710c05 Refactor source and bytecode file loaders in importlib so that there Brett Cannon <bcannon@gmail.com> parents: 51770 diff changeset	1057 def get_data(self, path):
ee7029710c05 Refactor source and bytecode file loaders in importlib so that there Brett Cannon <bcannon@gmail.com> parents: 51770 diff changeset	1058 """Return the data from path as raw bytes."""
64407 f8ba9558adbf Remove redundant context manager. Florent Xicluna <florent.xicluna@gmail.com> parents: 64291 diff changeset	1059 with _io.FileIO(path, 'r') as file:
62756 34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	1060 return file.read()
51771 ee7029710c05 Refactor source and bytecode file loaders in importlib so that there Brett Cannon <bcannon@gmail.com> parents: 51770 diff changeset	1061
ee7029710c05 Refactor source and bytecode file loaders in importlib so that there Brett Cannon <bcannon@gmail.com> parents: 51770 diff changeset	1062
76478 1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and Brett Cannon <brett@python.org> parents: 76472 diff changeset	1063 class SourceFileLoader(FileLoader, SourceLoader):
51771 ee7029710c05 Refactor source and bytecode file loaders in importlib so that there Brett Cannon <bcannon@gmail.com> parents: 51770 diff changeset	1064
62756 34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	1065 """Concrete implementation of SourceLoader using the file system."""
51771 ee7029710c05 Refactor source and bytecode file loaders in importlib so that there Brett Cannon <bcannon@gmail.com> parents: 51770 diff changeset	1066
74358 87331661042b Issue #13645: pyc files now contain the size of the corresponding source Antoine Pitrou <solipsis@pitrou.net> parents: 73977 diff changeset	1067 def path_stats(self, path):
78491 f7b59e890e30 fix docstring wording Philip Jenvey <pjenvey@underboss.org> parents: 78489 diff changeset	1068 """Return the metadata for the path."""
74358 87331661042b Issue #13645: pyc files now contain the size of the corresponding source Antoine Pitrou <solipsis@pitrou.net> parents: 73977 diff changeset	1069 st = _os.stat(path)
87331661042b Issue #13645: pyc files now contain the size of the corresponding source Antoine Pitrou <solipsis@pitrou.net> parents: 73977 diff changeset	1070 return {'mtime': st.st_mtime, 'size': st.st_size}
51771 ee7029710c05 Refactor source and bytecode file loaders in importlib so that there Brett Cannon <bcannon@gmail.com> parents: 51770 diff changeset	1071
78722 3a831a0a29c4 Close #2501: Permission bits are once again correctly copied from the source file to the cached bytecode file. Test by Eric Snow. Nick Coghlan <ncoghlan@gmail.com> parents: 78663 diff changeset	1072 def _cache_bytecode(self, source_path, bytecode_path, data):
3a831a0a29c4 Close #2501: Permission bits are once again correctly copied from the source file to the cached bytecode file. Test by Eric Snow. Nick Coghlan <ncoghlan@gmail.com> parents: 78663 diff changeset	1073 # Adapt between the two APIs
83376 04aaeae6ee7b Issue #17244: Don't mask exceptions raised during the creation of Brett Cannon <brett@python.org> parents: 83225 diff changeset	1074 mode = _calc_mode(source_path)
78738 0c661c5632e0 Issue #2051: Tweak last commit for this issue to pass in mode instead Brett Cannon <brett@python.org> parents: 78722 diff changeset	1075 return self.set_data(bytecode_path, data, _mode=mode)
78722 3a831a0a29c4 Close #2501: Permission bits are once again correctly copied from the source file to the cached bytecode file. Test by Eric Snow. Nick Coghlan <ncoghlan@gmail.com> parents: 78663 diff changeset	1076
78738 0c661c5632e0 Issue #2051: Tweak last commit for this issue to pass in mode instead Brett Cannon <brett@python.org> parents: 78722 diff changeset	1077 def set_data(self, path, data, *, _mode=0o666):
62587 b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	1078 """Write bytes data to a file."""
76449 163360370ad2 Make path manipulation more robust for platforms with alternative path Brett Cannon <brett@python.org> parents: 76445 diff changeset	1079 parent, filename = _path_split(path)
64240 cf399de12b54 One of the joys of having test_multiprocessing occasionally execute after Brett Cannon <bcannon@gmail.com> parents: 62761 diff changeset	1080 path_parts = []
cf399de12b54 One of the joys of having test_multiprocessing occasionally execute after Brett Cannon <bcannon@gmail.com> parents: 62761 diff changeset	1081 # Figure out what directories are missing.
cf399de12b54 One of the joys of having test_multiprocessing occasionally execute after Brett Cannon <bcannon@gmail.com> parents: 62761 diff changeset	1082 while parent and not _path_isdir(parent):
76449 163360370ad2 Make path manipulation more robust for platforms with alternative path Brett Cannon <brett@python.org> parents: 76445 diff changeset	1083 parent, part = _path_split(parent)
64240 cf399de12b54 One of the joys of having test_multiprocessing occasionally execute after Brett Cannon <bcannon@gmail.com> parents: 62761 diff changeset	1084 path_parts.append(part)
cf399de12b54 One of the joys of having test_multiprocessing occasionally execute after Brett Cannon <bcannon@gmail.com> parents: 62761 diff changeset	1085 # Create needed directories.
cf399de12b54 One of the joys of having test_multiprocessing occasionally execute after Brett Cannon <bcannon@gmail.com> parents: 62761 diff changeset	1086 for part in reversed(path_parts):
cf399de12b54 One of the joys of having test_multiprocessing occasionally execute after Brett Cannon <bcannon@gmail.com> parents: 62761 diff changeset	1087 parent = _path_join(parent, part)
cf399de12b54 One of the joys of having test_multiprocessing occasionally execute after Brett Cannon <bcannon@gmail.com> parents: 62761 diff changeset	1088 try:
cf399de12b54 One of the joys of having test_multiprocessing occasionally execute after Brett Cannon <bcannon@gmail.com> parents: 62761 diff changeset	1089 _os.mkdir(parent)
73169 e4d44c2e8e81 Simplify and remove few dependencies on 'errno', thanks to PEP 3151. Florent Xicluna <florent.xicluna@gmail.com> parents: 73016 diff changeset	1090 except FileExistsError:
64240 cf399de12b54 One of the joys of having test_multiprocessing occasionally execute after Brett Cannon <bcannon@gmail.com> parents: 62761 diff changeset	1091 # Probably another Python process already created the dir.
73169 e4d44c2e8e81 Simplify and remove few dependencies on 'errno', thanks to PEP 3151. Florent Xicluna <florent.xicluna@gmail.com> parents: 73016 diff changeset	1092 continue
79741 53ce30b62de2 Issue #15833: don't raise an exception if importlib can't write byte-compiled Trent Nelson <trent@trent.me> parents: 79652 diff changeset	1093 except OSError as exc:
53ce30b62de2 Issue #15833: don't raise an exception if importlib can't write byte-compiled Trent Nelson <trent@trent.me> parents: 79652 diff changeset	1094 # Could be a permission error, read-only filesystem: just forget
53ce30b62de2 Issue #15833: don't raise an exception if importlib can't write byte-compiled Trent Nelson <trent@trent.me> parents: 79652 diff changeset	1095 # about writing the data.
53ce30b62de2 Issue #15833: don't raise an exception if importlib can't write byte-compiled Trent Nelson <trent@trent.me> parents: 79652 diff changeset	1096 _verbose_message('could not create {!r}: {!r}', parent, exc)
73169 e4d44c2e8e81 Simplify and remove few dependencies on 'errno', thanks to PEP 3151. Florent Xicluna <florent.xicluna@gmail.com> parents: 73016 diff changeset	1097 return
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	1098 try:
78738 0c661c5632e0 Issue #2051: Tweak last commit for this issue to pass in mode instead Brett Cannon <brett@python.org> parents: 78722 diff changeset	1099 _write_atomic(path, data, _mode)
76753 59b8a7fcb047 Properly mark names in importlib._bootstrap as private. Brett Cannon <brett@python.org> parents: 76586 diff changeset	1100 _verbose_message('created {!r}', path)
79741 53ce30b62de2 Issue #15833: don't raise an exception if importlib can't write byte-compiled Trent Nelson <trent@trent.me> parents: 79652 diff changeset	1101 except OSError as exc:
53ce30b62de2 Issue #15833: don't raise an exception if importlib can't write byte-compiled Trent Nelson <trent@trent.me> parents: 79652 diff changeset	1102 # Same as above: just don't write the bytecode.
53ce30b62de2 Issue #15833: don't raise an exception if importlib can't write byte-compiled Trent Nelson <trent@trent.me> parents: 79652 diff changeset	1103 _verbose_message('could not create {!r}: {!r}', path, exc)
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	1104
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	1105
76548 a2cf07135e4f Issue #14605: Rename _SourcelessFileLoader to SourcelessFileLoader. Marc-Andre Lemburg <mal@egenix.com> parents: 76547 diff changeset	1106 class SourcelessFileLoader(FileLoader, _LoaderBasics):
55355 dfaf6de2c8ce Implement the PEP 302 protocol for get_filename() as Brett Cannon <bcannon@gmail.com> parents: 52633 diff changeset	1107
62756 34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	1108 """Loader which handles sourceless file imports."""
52112 03080fa08dfe Implement get_source for importlib.abc.PyLoader using source_path and get_data. Brett Cannon <bcannon@gmail.com> parents: 52083 diff changeset	1109
84001 e873f2e67353 Issues #18088, 18089: Introduce Brett Cannon <brett@python.org> parents: 83999 diff changeset	1110 def init_module_attrs(self, module):
e873f2e67353 Issues #18088, 18089: Introduce Brett Cannon <brett@python.org> parents: 83999 diff changeset	1111 super().init_module_attrs(module)
e873f2e67353 Issues #18088, 18089: Introduce Brett Cannon <brett@python.org> parents: 83999 diff changeset	1112 module.__cached__ = module.__file__
51771 ee7029710c05 Refactor source and bytecode file loaders in importlib so that there Brett Cannon <bcannon@gmail.com> parents: 51770 diff changeset	1113
ee7029710c05 Refactor source and bytecode file loaders in importlib so that there Brett Cannon <bcannon@gmail.com> parents: 51770 diff changeset	1114 def get_code(self, fullname):
62756 34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	1115 path = self.get_filename(fullname)
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	1116 data = self.get_data(path)
81432 a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	1117 bytes_data = _validate_bytecode_header(data, name=fullname, path=path)
a4292889e942 Issue #15031: Refactor some code in importlib pertaining to validating Brett Cannon <brett@python.org> parents: 81428 diff changeset	1118 return _compile_bytecode(bytes_data, name=fullname, bytecode_path=path)
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	1119
62756 34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	1120 def get_source(self, fullname):
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	1121 """Return None as there is no source code."""
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	1122 return None
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	1123
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	1124
78486 1db6553f3f8c Issue #15576: Allow extension modules to be a package's __init__ Brett Cannon <brett@python.org> parents: 78485 diff changeset	1125 # Filled in by _setup().
1db6553f3f8c Issue #15576: Allow extension modules to be a package's __init__ Brett Cannon <brett@python.org> parents: 78485 diff changeset	1126 EXTENSION_SUFFIXES = []
1db6553f3f8c Issue #15576: Allow extension modules to be a package's __init__ Brett Cannon <brett@python.org> parents: 78485 diff changeset	1127
1db6553f3f8c Issue #15576: Allow extension modules to be a package's __init__ Brett Cannon <brett@python.org> parents: 78485 diff changeset	1128
76478 1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and Brett Cannon <brett@python.org> parents: 76472 diff changeset	1129 class ExtensionFileLoader:
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	1130
52141 f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1131 """Loader for extension modules.
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	1132
52141 f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1133 The constructor is designed to work with FileFinder.
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	1134
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	1135 """
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	1136
62756 34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	1137 def __init__(self, name, path):
76478 1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and Brett Cannon <brett@python.org> parents: 76472 diff changeset	1138 self.name = name
1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and Brett Cannon <brett@python.org> parents: 76472 diff changeset	1139 self.path = path
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	1140
52141 f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1141 @_check_name
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1142 @set_package
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1143 @set_loader
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1144 def load_module(self, fullname):
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1145 """Load an extension module."""
84171 205aa49e5cd5 Issue #18115: Abstract out managing the cleanup of modules to use in Brett Cannon <brett@python.org> parents: 84169 diff changeset	1146 with _ManageReload(fullname):
78344 62033490ca0f Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise Nick Coghlan <ncoghlan@gmail.com> parents: 78314 diff changeset	1147 module = _call_with_frames_removed(_imp.load_dynamic,
62033490ca0f Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise Nick Coghlan <ncoghlan@gmail.com> parents: 78314 diff changeset	1148 fullname, self.path)
76753 59b8a7fcb047 Properly mark names in importlib._bootstrap as private. Brett Cannon <brett@python.org> parents: 76586 diff changeset	1149 _verbose_message('extension module loaded from {!r}', self.path)
78511 34d5ec8a1019 Don't overwrite a __path__ value from extension modules if already Brett Cannon <brett@python.org> parents: 78491 diff changeset	1150 if self.is_package(fullname) and not hasattr(module, '__path__'):
78486 1db6553f3f8c Issue #15576: Allow extension modules to be a package's __init__ Brett Cannon <brett@python.org> parents: 78485 diff changeset	1151 module.__path__ = [_path_split(self.path)[0]]
76303 2dd046be2c88 Issue #2377: Make importlib the implementation of __import__(). Brett Cannon <brett@python.org> parents: 76280 diff changeset	1152 return module
52141 f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1153
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1154 def is_package(self, fullname):
78491 f7b59e890e30 fix docstring wording Philip Jenvey <pjenvey@underboss.org> parents: 78489 diff changeset	1155 """Return True if the extension module is a package."""
78486 1db6553f3f8c Issue #15576: Allow extension modules to be a package's __init__ Brett Cannon <brett@python.org> parents: 78485 diff changeset	1156 file_name = _path_split(self.path)[1]
78487 e024f6ba5ed8 update docstring per the extension package fix, refactor Philip Jenvey <pjenvey@underboss.org> parents: 78486 diff changeset	1157 return any(file_name == '__init__' + suffix
e024f6ba5ed8 update docstring per the extension package fix, refactor Philip Jenvey <pjenvey@underboss.org> parents: 78486 diff changeset	1158 for suffix in EXTENSION_SUFFIXES)
52141 f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1159
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1160 def get_code(self, fullname):
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1161 """Return None as an extension module cannot create a code object."""
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1162 return None
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1163
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1164 def get_source(self, fullname):
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1165 """Return None as extension modules have no source code."""
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1166 return None
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	1167
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	1168
77126 702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1169 class _NamespacePath:
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1170 """Represents a namespace package's path. It uses the module name
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1171 to find its parent module, and from there it looks up the parent's
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1172 __path__. When this changes, the module's own path is recomputed,
80409 58d6fea0c0c7 missing letter Benjamin Peterson <benjamin@python.org> parents: 79832 diff changeset	1173 using path_finder. For top-level modules, the parent module's path
77126 702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1174 is sys.path."""
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1175
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1176 def __init__(self, name, path, path_finder):
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1177 self._name = name
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1178 self._path = path
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1179 self._last_parent_path = tuple(self._get_parent_path())
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1180 self._path_finder = path_finder
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1181
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1182 def _find_parent_path_names(self):
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1183 """Returns a tuple of (parent-module-name, parent-path-attr-name)"""
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1184 parent, dot, me = self._name.rpartition('.')
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1185 if dot == '':
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1186 # This is a top-level module. sys.path contains the parent path.
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1187 return 'sys', 'path'
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1188 # Not a top-level module. parent-module.__path__ contains the
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1189 # parent path.
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1190 return parent, '__path__'
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1191
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1192 def _get_parent_path(self):
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1193 parent_module_name, path_attr_name = self._find_parent_path_names()
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1194 return getattr(sys.modules[parent_module_name], path_attr_name)
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1195
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1196 def _recalculate(self):
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1197 # If the parent's path has changed, recalculate _path
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1198 parent_path = tuple(self._get_parent_path()) # Make a copy
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1199 if parent_path != self._last_parent_path:
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1200 loader, new_path = self._path_finder(self._name, parent_path)
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1201 # Note that no changes are made if a loader is returned, but we
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1202 # do remember the new parent path
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1203 if loader is None:
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1204 self._path = new_path
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1205 self._last_parent_path = parent_path # Save the copy
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1206 return self._path
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1207
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1208 def __iter__(self):
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1209 return iter(self._recalculate())
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1210
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1211 def __len__(self):
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1212 return len(self._recalculate())
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1213
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1214 def __repr__(self):
77964 37850f083af5 Minor refactoring in importlib._bootstrap, and fix the '_wrap' docstring. Florent Xicluna <florent.xicluna@gmail.com> parents: 77917 diff changeset	1215 return "_NamespacePath({!r})".format(self._path)
77126 702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1216
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1217 def __contains__(self, item):
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1218 return item in self._recalculate()
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1219
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1220 def append(self, item):
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1221 self._path.append(item)
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1222
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1223
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1224 class NamespaceLoader:
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1225 def __init__(self, name, path, path_finder):
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1226 self._path = _NamespacePath(name, path, path_finder)
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1227
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1228 @classmethod
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1229 def module_repr(cls, module):
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1230 return "<module '{}' (namespace)>".format(module.__name__)
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1231
84169 ebec625b13f9 Issues #18058, 18057: Make importlib._bootstrap.NamespaceLoader Brett Cannon <brett@python.org> parents: 84134 diff changeset	1232 def is_package(self, fullname):
ebec625b13f9 Issues #18058, 18057: Make importlib._bootstrap.NamespaceLoader Brett Cannon <brett@python.org> parents: 84134 diff changeset	1233 return True
ebec625b13f9 Issues #18058, 18057: Make importlib._bootstrap.NamespaceLoader Brett Cannon <brett@python.org> parents: 84134 diff changeset	1234
ebec625b13f9 Issues #18058, 18057: Make importlib._bootstrap.NamespaceLoader Brett Cannon <brett@python.org> parents: 84134 diff changeset	1235 def get_source(self, fullname):
ebec625b13f9 Issues #18058, 18057: Make importlib._bootstrap.NamespaceLoader Brett Cannon <brett@python.org> parents: 84134 diff changeset	1236 return ''
ebec625b13f9 Issues #18058, 18057: Make importlib._bootstrap.NamespaceLoader Brett Cannon <brett@python.org> parents: 84134 diff changeset	1237
ebec625b13f9 Issues #18058, 18057: Make importlib._bootstrap.NamespaceLoader Brett Cannon <brett@python.org> parents: 84134 diff changeset	1238 def get_code(self, fullname):
ebec625b13f9 Issues #18058, 18057: Make importlib._bootstrap.NamespaceLoader Brett Cannon <brett@python.org> parents: 84134 diff changeset	1239 return compile('', '<string>', 'exec', dont_inherit=True)
ebec625b13f9 Issues #18058, 18057: Make importlib._bootstrap.NamespaceLoader Brett Cannon <brett@python.org> parents: 84134 diff changeset	1240
ebec625b13f9 Issues #18058, 18057: Make importlib._bootstrap.NamespaceLoader Brett Cannon <brett@python.org> parents: 84134 diff changeset	1241 def init_module_attrs(self, module):
ebec625b13f9 Issues #18058, 18057: Make importlib._bootstrap.NamespaceLoader Brett Cannon <brett@python.org> parents: 84134 diff changeset	1242 module.__loader__ = self
ebec625b13f9 Issues #18058, 18057: Make importlib._bootstrap.NamespaceLoader Brett Cannon <brett@python.org> parents: 84134 diff changeset	1243 module.__package__ = module.__name__
ebec625b13f9 Issues #18058, 18057: Make importlib._bootstrap.NamespaceLoader Brett Cannon <brett@python.org> parents: 84134 diff changeset	1244
84001 e873f2e67353 Issues #18088, 18089: Introduce Brett Cannon <brett@python.org> parents: 83999 diff changeset	1245 def load_module(self, fullname):
77126 702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1246 """Load a namespace module."""
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1247 _verbose_message('namespace module loaded with path {!r}', self._path)
84001 e873f2e67353 Issues #18088, 18089: Introduce Brett Cannon <brett@python.org> parents: 83999 diff changeset	1248 with module_to_load(fullname) as module:
84169 ebec625b13f9 Issues #18058, 18057: Make importlib._bootstrap.NamespaceLoader Brett Cannon <brett@python.org> parents: 84134 diff changeset	1249 self.init_module_attrs(module)
84001 e873f2e67353 Issues #18088, 18089: Introduce Brett Cannon <brett@python.org> parents: 83999 diff changeset	1250 module.__path__ = self._path
e873f2e67353 Issues #18088, 18089: Introduce Brett Cannon <brett@python.org> parents: 83999 diff changeset	1251 return module
77126 702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1252
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1253
52141 f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1254 # Finders #####################################################################
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	1255
51322 53d6445c6c5a Rename importlib._bootstrap.SysPathFinder to PathFinder and expose off of Brett Cannon <bcannon@gmail.com> parents: 51266 diff changeset	1256 class PathFinder:
51266 63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to Brett Cannon <bcannon@gmail.com> parents: 51242 diff changeset	1257
78382 1f8351cf00f3 Issue #15502: Bring the importlib.PathFinder docs and docstring more in line with the new import system documentation, and fix various parts of the new docs that weren't quite right given PEP 420 or were otherwise a bit misleading. Also note the key terminology problem still being discussed in the issue Nick Coghlan <ncoghlan@gmail.com> parents: 78380 diff changeset	1258 """Meta path finder for sys.path and package __path__ attributes."""
51266 63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to Brett Cannon <bcannon@gmail.com> parents: 51242 diff changeset	1259
51639 ef2cba80c975 Change importlib.machinery.PathFinder to not have implicit semantics (that's Brett Cannon <bcannon@gmail.com> parents: 51433 diff changeset	1260 @classmethod
78485 0a75ce232f56 Issue #15502: Finish bringing importlib.abc in line with the current Brett Cannon <brett@python.org> parents: 78456 diff changeset	1261 def invalidate_caches(cls):
0a75ce232f56 Issue #15502: Finish bringing importlib.abc in line with the current Brett Cannon <brett@python.org> parents: 78456 diff changeset	1262 """Call the invalidate_caches() method on all path entry finders
0a75ce232f56 Issue #15502: Finish bringing importlib.abc in line with the current Brett Cannon <brett@python.org> parents: 78456 diff changeset	1263 stored in sys.path_importer_caches (where implemented)."""
0a75ce232f56 Issue #15502: Finish bringing importlib.abc in line with the current Brett Cannon <brett@python.org> parents: 78456 diff changeset	1264 for finder in sys.path_importer_cache.values():
0a75ce232f56 Issue #15502: Finish bringing importlib.abc in line with the current Brett Cannon <brett@python.org> parents: 78456 diff changeset	1265 if hasattr(finder, 'invalidate_caches'):
0a75ce232f56 Issue #15502: Finish bringing importlib.abc in line with the current Brett Cannon <brett@python.org> parents: 78456 diff changeset	1266 finder.invalidate_caches()
0a75ce232f56 Issue #15502: Finish bringing importlib.abc in line with the current Brett Cannon <brett@python.org> parents: 78456 diff changeset	1267
0a75ce232f56 Issue #15502: Finish bringing importlib.abc in line with the current Brett Cannon <brett@python.org> parents: 78456 diff changeset	1268 @classmethod
76566 57d558f1904d Issue #14605: Make explicit the entries on sys.path_hooks that used to Brett Cannon <brett@python.org> parents: 76549 diff changeset	1269 def _path_hooks(cls, path):
51639 ef2cba80c975 Change importlib.machinery.PathFinder to not have implicit semantics (that's Brett Cannon <bcannon@gmail.com> parents: 51433 diff changeset	1270 """Search sequence of hooks for a finder for 'path'.
51266 63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to Brett Cannon <bcannon@gmail.com> parents: 51242 diff changeset	1271
51639 ef2cba80c975 Change importlib.machinery.PathFinder to not have implicit semantics (that's Brett Cannon <bcannon@gmail.com> parents: 51433 diff changeset	1272 If 'hooks' is false then use sys.path_hooks.
51266 63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to Brett Cannon <bcannon@gmail.com> parents: 51242 diff changeset	1273
63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to Brett Cannon <bcannon@gmail.com> parents: 51242 diff changeset	1274 """
76566 57d558f1904d Issue #14605: Make explicit the entries on sys.path_hooks that used to Brett Cannon <brett@python.org> parents: 76549 diff changeset	1275 if not sys.path_hooks:
57d558f1904d Issue #14605: Make explicit the entries on sys.path_hooks that used to Brett Cannon <brett@python.org> parents: 76549 diff changeset	1276 _warnings.warn('sys.path_hooks is empty', ImportWarning)
57d558f1904d Issue #14605: Make explicit the entries on sys.path_hooks that used to Brett Cannon <brett@python.org> parents: 76549 diff changeset	1277 for hook in sys.path_hooks:
51266 63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to Brett Cannon <bcannon@gmail.com> parents: 51242 diff changeset	1278 try:
63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to Brett Cannon <bcannon@gmail.com> parents: 51242 diff changeset	1279 return hook(path)
63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to Brett Cannon <bcannon@gmail.com> parents: 51242 diff changeset	1280 except ImportError:
63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to Brett Cannon <bcannon@gmail.com> parents: 51242 diff changeset	1281 continue
63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to Brett Cannon <bcannon@gmail.com> parents: 51242 diff changeset	1282 else:
76582 7025ee00dbf6 Issue #14605: Use None in sys.path_importer_cache to represent no Brett Cannon <brett@python.org> parents: 76579 diff changeset	1283 return None
51266 63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to Brett Cannon <bcannon@gmail.com> parents: 51242 diff changeset	1284
51322 53d6445c6c5a Rename importlib._bootstrap.SysPathFinder to PathFinder and expose off of Brett Cannon <bcannon@gmail.com> parents: 51266 diff changeset	1285 @classmethod
76566 57d558f1904d Issue #14605: Make explicit the entries on sys.path_hooks that used to Brett Cannon <brett@python.org> parents: 76549 diff changeset	1286 def _path_importer_cache(cls, path):
78382 1f8351cf00f3 Issue #15502: Bring the importlib.PathFinder docs and docstring more in line with the new import system documentation, and fix various parts of the new docs that weren't quite right given PEP 420 or were otherwise a bit misleading. Also note the key terminology problem still being discussed in the issue Nick Coghlan <ncoghlan@gmail.com> parents: 78380 diff changeset	1287 """Get the finder for the path entry from sys.path_importer_cache.
51266 63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to Brett Cannon <bcannon@gmail.com> parents: 51242 diff changeset	1288
78382 1f8351cf00f3 Issue #15502: Bring the importlib.PathFinder docs and docstring more in line with the new import system documentation, and fix various parts of the new docs that weren't quite right given PEP 420 or were otherwise a bit misleading. Also note the key terminology problem still being discussed in the issue Nick Coghlan <ncoghlan@gmail.com> parents: 78380 diff changeset	1289 If the path entry is not in the cache, find the appropriate finder
1f8351cf00f3 Issue #15502: Bring the importlib.PathFinder docs and docstring more in line with the new import system documentation, and fix various parts of the new docs that weren't quite right given PEP 420 or were otherwise a bit misleading. Also note the key terminology problem still being discussed in the issue Nick Coghlan <ncoghlan@gmail.com> parents: 78380 diff changeset	1290 and cache it. If no finder is available, store None.
51266 63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to Brett Cannon <bcannon@gmail.com> parents: 51242 diff changeset	1291
63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to Brett Cannon <bcannon@gmail.com> parents: 51242 diff changeset	1292 """
74994 b8593ec7e8c5 Tweak the handling of the empty string in sys.path for importlib. Brett Cannon <brett@python.org> parents: 74992 diff changeset	1293 if path == '':
75048 bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function. Antoine Pitrou <solipsis@pitrou.net> parents: 75045 diff changeset	1294 path = '.'
51266 63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to Brett Cannon <bcannon@gmail.com> parents: 51242 diff changeset	1295 try:
51322 53d6445c6c5a Rename importlib._bootstrap.SysPathFinder to PathFinder and expose off of Brett Cannon <bcannon@gmail.com> parents: 51266 diff changeset	1296 finder = sys.path_importer_cache[path]
51266 63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to Brett Cannon <bcannon@gmail.com> parents: 51242 diff changeset	1297 except KeyError:
51322 53d6445c6c5a Rename importlib._bootstrap.SysPathFinder to PathFinder and expose off of Brett Cannon <bcannon@gmail.com> parents: 51266 diff changeset	1298 finder = cls._path_hooks(path)
51266 63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to Brett Cannon <bcannon@gmail.com> parents: 51242 diff changeset	1299 sys.path_importer_cache[path] = finder
63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to Brett Cannon <bcannon@gmail.com> parents: 51242 diff changeset	1300 return finder
63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to Brett Cannon <bcannon@gmail.com> parents: 51242 diff changeset	1301
51322 53d6445c6c5a Rename importlib._bootstrap.SysPathFinder to PathFinder and expose off of Brett Cannon <bcannon@gmail.com> parents: 51266 diff changeset	1302 @classmethod
77126 702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1303 def _get_loader(cls, fullname, path):
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1304 """Find the loader or namespace_path for this module/package name."""
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1305 # If this ends up being a namespace package, namespace_path is
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1306 # the list of paths that will become its __path__
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1307 namespace_path = []
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1308 for entry in path:
80529 291406748217 - Issue #16514: Fix regression causing a traceback when sys.path[0] is None Barry Warsaw <barry@python.org> parents: 80409 diff changeset	1309 if not isinstance(entry, (str, bytes)):
291406748217 - Issue #16514: Fix regression causing a traceback when sys.path[0] is None Barry Warsaw <barry@python.org> parents: 80409 diff changeset	1310 continue
77126 702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1311 finder = cls._path_importer_cache(entry)
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1312 if finder is not None:
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1313 if hasattr(finder, 'find_loader'):
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1314 loader, portions = finder.find_loader(fullname)
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1315 else:
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1316 loader = finder.find_module(fullname)
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1317 portions = []
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1318 if loader is not None:
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1319 # We found a loader: return it immediately.
78485 0a75ce232f56 Issue #15502: Finish bringing importlib.abc in line with the current Brett Cannon <brett@python.org> parents: 78456 diff changeset	1320 return loader, namespace_path
77126 702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1321 # This is possibly part of a namespace package.
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1322 # Remember these path entries (if any) for when we
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1323 # create a namespace package, and continue iterating
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1324 # on path.
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1325 namespace_path.extend(portions)
51266 63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to Brett Cannon <bcannon@gmail.com> parents: 51242 diff changeset	1326 else:
78485 0a75ce232f56 Issue #15502: Finish bringing importlib.abc in line with the current Brett Cannon <brett@python.org> parents: 78456 diff changeset	1327 return None, namespace_path
51266 63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to Brett Cannon <bcannon@gmail.com> parents: 51242 diff changeset	1328
51322 53d6445c6c5a Rename importlib._bootstrap.SysPathFinder to PathFinder and expose off of Brett Cannon <bcannon@gmail.com> parents: 51266 diff changeset	1329 @classmethod
53d6445c6c5a Rename importlib._bootstrap.SysPathFinder to PathFinder and expose off of Brett Cannon <bcannon@gmail.com> parents: 51266 diff changeset	1330 def find_module(cls, fullname, path=None):
52165 62f9dad628de A few more docstring/API cleanups for importlib. Brett Cannon <bcannon@gmail.com> parents: 52164 diff changeset	1331 """Find the module on sys.path or 'path' based on sys.path_hooks and
62f9dad628de A few more docstring/API cleanups for importlib. Brett Cannon <bcannon@gmail.com> parents: 52164 diff changeset	1332 sys.path_importer_cache."""
76384 c1399cf7bd6a Issue #12599: Be more strict in accepting None vs. a false-like object Brett Cannon <brett@python.org> parents: 76331 diff changeset	1333 if path is None:
51266 63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to Brett Cannon <bcannon@gmail.com> parents: 51242 diff changeset	1334 path = sys.path
77126 702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1335 loader, namespace_path = cls._get_loader(fullname, path)
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1336 if loader is not None:
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1337 return loader
51266 63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to Brett Cannon <bcannon@gmail.com> parents: 51242 diff changeset	1338 else:
77126 702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1339 if namespace_path:
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1340 # We found at least one namespace path. Return a
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1341 # loader which can create the namespace package.
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1342 return NamespaceLoader(fullname, namespace_path, cls._get_loader)
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1343 else:
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1344 return None
51266 63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to Brett Cannon <bcannon@gmail.com> parents: 51242 diff changeset	1345
63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to Brett Cannon <bcannon@gmail.com> parents: 51242 diff changeset	1346
76478 1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and Brett Cannon <brett@python.org> parents: 76472 diff changeset	1347 class FileFinder:
52141 f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1348
62756 34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	1349 """File-based finder.
52141 f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1350
76478 1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and Brett Cannon <brett@python.org> parents: 76472 diff changeset	1351 Interactions with the file system are cached for performance, being
1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and Brett Cannon <brett@python.org> parents: 76472 diff changeset	1352 refreshed when the directory the finder is handling has been modified.
52141 f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1353
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1354 """
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1355
62756 34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	1356 def __init__(self, path, *details):
76478 1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and Brett Cannon <brett@python.org> parents: 76472 diff changeset	1357 """Initialize with the path to search on and a variable number of
82236 0f65bf6063ca Fixes a FileFinder docstring to reflect an old change. Eric Snow <ericsnowcurrently@gmail.com> parents: 81884 diff changeset	1358 2-tuples containing the loader and the file suffixes the loader
0f65bf6063ca Fixes a FileFinder docstring to reflect an old change. Eric Snow <ericsnowcurrently@gmail.com> parents: 81884 diff changeset	1359 recognizes."""
78486 1db6553f3f8c Issue #15576: Allow extension modules to be a package's __init__ Brett Cannon <brett@python.org> parents: 78485 diff changeset	1360 loaders = []
1db6553f3f8c Issue #15576: Allow extension modules to be a package's __init__ Brett Cannon <brett@python.org> parents: 78485 diff changeset	1361 for loader, suffixes in details:
1db6553f3f8c Issue #15576: Allow extension modules to be a package's __init__ Brett Cannon <brett@python.org> parents: 78485 diff changeset	1362 loaders.extend((suffix, loader) for suffix in suffixes)
1db6553f3f8c Issue #15576: Allow extension modules to be a package's __init__ Brett Cannon <brett@python.org> parents: 78485 diff changeset	1363 self._loaders = loaders
75048 bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function. Antoine Pitrou <solipsis@pitrou.net> parents: 75045 diff changeset	1364 # Base (directory) path
bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function. Antoine Pitrou <solipsis@pitrou.net> parents: 75045 diff changeset	1365 self.path = path or '.'
bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function. Antoine Pitrou <solipsis@pitrou.net> parents: 75045 diff changeset	1366 self._path_mtime = -1
bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function. Antoine Pitrou <solipsis@pitrou.net> parents: 75045 diff changeset	1367 self._path_cache = set()
75082 3297dcdad196 Issue #14063: fix test_importlib failure under OS X case-insensitive filesystems Antoine Pitrou <solipsis@pitrou.net> parents: 75070 diff changeset	1368 self._relaxed_path_cache = set()
52141 f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1369
75324 de125ee32897 Update importlib.invalidate_caches() to be more general. Brett Cannon <brett@python.org> parents: 75250 diff changeset	1370 def invalidate_caches(self):
de125ee32897 Update importlib.invalidate_caches() to be more general. Brett Cannon <brett@python.org> parents: 75250 diff changeset	1371 """Invalidate the directory mtime."""
de125ee32897 Update importlib.invalidate_caches() to be more general. Brett Cannon <brett@python.org> parents: 75250 diff changeset	1372 self._path_mtime = -1
52141 f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1373
78489 e7a67f1bf604 Issue #15502: Refactor some code. Brett Cannon <brett@python.org> parents: 78487 diff changeset	1374 find_module = _find_module_shim
77126 702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1375
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1376 def find_loader(self, fullname):
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1377 """Try to find a loader for the specified module, or the namespace
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1378 package portions. Returns (loader, list-of-portions)."""
77757 2051fead1933 Fixes issue 15039: namespace packages are no longer imported in preference to modules of the same name. Eric V. Smith <eric@trueblade.com> parents: 77600 diff changeset	1379 is_namespace = False
52141 f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1380 tail_module = fullname.rpartition('.')[2]
75048 bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function. Antoine Pitrou <solipsis@pitrou.net> parents: 75045 diff changeset	1381 try:
bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function. Antoine Pitrou <solipsis@pitrou.net> parents: 75045 diff changeset	1382 mtime = _os.stat(self.path).st_mtime
bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function. Antoine Pitrou <solipsis@pitrou.net> parents: 75045 diff changeset	1383 except OSError:
bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function. Antoine Pitrou <solipsis@pitrou.net> parents: 75045 diff changeset	1384 mtime = -1
75324 de125ee32897 Update importlib.invalidate_caches() to be more general. Brett Cannon <brett@python.org> parents: 75250 diff changeset	1385 if mtime != self._path_mtime:
75048 bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function. Antoine Pitrou <solipsis@pitrou.net> parents: 75045 diff changeset	1386 self._fill_cache()
bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function. Antoine Pitrou <solipsis@pitrou.net> parents: 75045 diff changeset	1387 self._path_mtime = mtime
75082 3297dcdad196 Issue #14063: fix test_importlib failure under OS X case-insensitive filesystems Antoine Pitrou <solipsis@pitrou.net> parents: 75070 diff changeset	1388 # tail_module keeps the original casing, for __file__ and friends
3297dcdad196 Issue #14063: fix test_importlib failure under OS X case-insensitive filesystems Antoine Pitrou <solipsis@pitrou.net> parents: 75070 diff changeset	1389 if _relax_case():
3297dcdad196 Issue #14063: fix test_importlib failure under OS X case-insensitive filesystems Antoine Pitrou <solipsis@pitrou.net> parents: 75070 diff changeset	1390 cache = self._relaxed_path_cache
3297dcdad196 Issue #14063: fix test_importlib failure under OS X case-insensitive filesystems Antoine Pitrou <solipsis@pitrou.net> parents: 75070 diff changeset	1391 cache_module = tail_module.lower()
3297dcdad196 Issue #14063: fix test_importlib failure under OS X case-insensitive filesystems Antoine Pitrou <solipsis@pitrou.net> parents: 75070 diff changeset	1392 else:
3297dcdad196 Issue #14063: fix test_importlib failure under OS X case-insensitive filesystems Antoine Pitrou <solipsis@pitrou.net> parents: 75070 diff changeset	1393 cache = self._path_cache
3297dcdad196 Issue #14063: fix test_importlib failure under OS X case-insensitive filesystems Antoine Pitrou <solipsis@pitrou.net> parents: 75070 diff changeset	1394 cache_module = tail_module
76309 e2f48b076012 Add some comments. Brett Cannon <brett@python.org> parents: 76303 diff changeset	1395 # Check if the module is the name of a directory (and thus a package).
75082 3297dcdad196 Issue #14063: fix test_importlib failure under OS X case-insensitive filesystems Antoine Pitrou <solipsis@pitrou.net> parents: 75070 diff changeset	1396 if cache_module in cache:
75048 bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function. Antoine Pitrou <solipsis@pitrou.net> parents: 75045 diff changeset	1397 base_path = _path_join(self.path, tail_module)
bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function. Antoine Pitrou <solipsis@pitrou.net> parents: 75045 diff changeset	1398 if _path_isdir(base_path):
78486 1db6553f3f8c Issue #15576: Allow extension modules to be a package's __init__ Brett Cannon <brett@python.org> parents: 78485 diff changeset	1399 for suffix, loader in self._loaders:
75048 bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function. Antoine Pitrou <solipsis@pitrou.net> parents: 75045 diff changeset	1400 init_filename = '__init__' + suffix
bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function. Antoine Pitrou <solipsis@pitrou.net> parents: 75045 diff changeset	1401 full_path = _path_join(base_path, init_filename)
bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function. Antoine Pitrou <solipsis@pitrou.net> parents: 75045 diff changeset	1402 if _path_isfile(full_path):
77126 702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1403 return (loader(fullname, full_path), [base_path])
75048 bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function. Antoine Pitrou <solipsis@pitrou.net> parents: 75045 diff changeset	1404 else:
77757 2051fead1933 Fixes issue 15039: namespace packages are no longer imported in preference to modules of the same name. Eric V. Smith <eric@trueblade.com> parents: 77600 diff changeset	1405 # A namespace package, return the path if we don't also
2051fead1933 Fixes issue 15039: namespace packages are no longer imported in preference to modules of the same name. Eric V. Smith <eric@trueblade.com> parents: 77600 diff changeset	1406 # find a module in the next section.
2051fead1933 Fixes issue 15039: namespace packages are no longer imported in preference to modules of the same name. Eric V. Smith <eric@trueblade.com> parents: 77600 diff changeset	1407 is_namespace = True
76309 e2f48b076012 Add some comments. Brett Cannon <brett@python.org> parents: 76303 diff changeset	1408 # Check for a file w/ a proper suffix exists.
78486 1db6553f3f8c Issue #15576: Allow extension modules to be a package's __init__ Brett Cannon <brett@python.org> parents: 78485 diff changeset	1409 for suffix, loader in self._loaders:
83055 ac89e7bc0e2f Issue #17357: Add missing verbosity messages when running under Brett Cannon <brett@python.org> parents: 82236 diff changeset	1410 full_path = _path_join(self.path, tail_module + suffix)
ac89e7bc0e2f Issue #17357: Add missing verbosity messages when running under Brett Cannon <brett@python.org> parents: 82236 diff changeset	1411 _verbose_message('trying {}'.format(full_path), verbosity=2)
75082 3297dcdad196 Issue #14063: fix test_importlib failure under OS X case-insensitive filesystems Antoine Pitrou <solipsis@pitrou.net> parents: 75070 diff changeset	1412 if cache_module + suffix in cache:
75048 bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function. Antoine Pitrou <solipsis@pitrou.net> parents: 75045 diff changeset	1413 if _path_isfile(full_path):
77126 702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1414 return (loader(fullname, full_path), [])
77757 2051fead1933 Fixes issue 15039: namespace packages are no longer imported in preference to modules of the same name. Eric V. Smith <eric@trueblade.com> parents: 77600 diff changeset	1415 if is_namespace:
83055 ac89e7bc0e2f Issue #17357: Add missing verbosity messages when running under Brett Cannon <brett@python.org> parents: 82236 diff changeset	1416 _verbose_message('possible namespace for {}'.format(base_path))
77757 2051fead1933 Fixes issue 15039: namespace packages are no longer imported in preference to modules of the same name. Eric V. Smith <eric@trueblade.com> parents: 77600 diff changeset	1417 return (None, [base_path])
77126 702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1418 return (None, [])
62756 34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	1419
75048 bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function. Antoine Pitrou <solipsis@pitrou.net> parents: 75045 diff changeset	1420 def _fill_cache(self):
bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function. Antoine Pitrou <solipsis@pitrou.net> parents: 75045 diff changeset	1421 """Fill the cache of potential modules and packages for this directory."""
bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function. Antoine Pitrou <solipsis@pitrou.net> parents: 75045 diff changeset	1422 path = self.path
78663 bfd04bfb55c5 Close #14846: Handle a sys.path entry going away Nick Coghlan <ncoghlan@gmail.com> parents: 78619 diff changeset	1423 try:
bfd04bfb55c5 Close #14846: Handle a sys.path entry going away Nick Coghlan <ncoghlan@gmail.com> parents: 78619 diff changeset	1424 contents = _os.listdir(path)
81427 159967aa24a5 Issue #16730: Don't raise an exception in Brett Cannon <brett@python.org> parents: 80929 diff changeset	1425 except (FileNotFoundError, PermissionError, NotADirectoryError):
159967aa24a5 Issue #16730: Don't raise an exception in Brett Cannon <brett@python.org> parents: 80929 diff changeset	1426 # Directory has either been removed, turned into a file, or made
159967aa24a5 Issue #16730: Don't raise an exception in Brett Cannon <brett@python.org> parents: 80929 diff changeset	1427 # unreadable.
78663 bfd04bfb55c5 Close #14846: Handle a sys.path entry going away Nick Coghlan <ncoghlan@gmail.com> parents: 78619 diff changeset	1428 contents = []
75082 3297dcdad196 Issue #14063: fix test_importlib failure under OS X case-insensitive filesystems Antoine Pitrou <solipsis@pitrou.net> parents: 75070 diff changeset	1429 # We store two cached versions, to handle runtime changes of the
3297dcdad196 Issue #14063: fix test_importlib failure under OS X case-insensitive filesystems Antoine Pitrou <solipsis@pitrou.net> parents: 75070 diff changeset	1430 # PYTHONCASEOK environment variable.
76428 a32be109bd86 Issue #14581: Windows users are allowed to import modules w/o taking Brett Cannon <brett@python.org> parents: 76406 diff changeset	1431 if not sys.platform.startswith('win'):
a32be109bd86 Issue #14581: Windows users are allowed to import modules w/o taking Brett Cannon <brett@python.org> parents: 76406 diff changeset	1432 self._path_cache = set(contents)
a32be109bd86 Issue #14581: Windows users are allowed to import modules w/o taking Brett Cannon <brett@python.org> parents: 76406 diff changeset	1433 else:
a32be109bd86 Issue #14581: Windows users are allowed to import modules w/o taking Brett Cannon <brett@python.org> parents: 76406 diff changeset	1434 # Windows users can import modules with case-insensitive file
a32be109bd86 Issue #14581: Windows users are allowed to import modules w/o taking Brett Cannon <brett@python.org> parents: 76406 diff changeset	1435 # suffixes (for legacy reasons). Make the suffix lowercase here
a32be109bd86 Issue #14581: Windows users are allowed to import modules w/o taking Brett Cannon <brett@python.org> parents: 76406 diff changeset	1436 # so it's done once instead of for every import. This is safe as
a32be109bd86 Issue #14581: Windows users are allowed to import modules w/o taking Brett Cannon <brett@python.org> parents: 76406 diff changeset	1437 # the specified suffixes to check against are always specified in a
a32be109bd86 Issue #14581: Windows users are allowed to import modules w/o taking Brett Cannon <brett@python.org> parents: 76406 diff changeset	1438 # case-sensitive manner.
a32be109bd86 Issue #14581: Windows users are allowed to import modules w/o taking Brett Cannon <brett@python.org> parents: 76406 diff changeset	1439 lower_suffix_contents = set()
a32be109bd86 Issue #14581: Windows users are allowed to import modules w/o taking Brett Cannon <brett@python.org> parents: 76406 diff changeset	1440 for item in contents:
a32be109bd86 Issue #14581: Windows users are allowed to import modules w/o taking Brett Cannon <brett@python.org> parents: 76406 diff changeset	1441 name, dot, suffix = item.partition('.')
a32be109bd86 Issue #14581: Windows users are allowed to import modules w/o taking Brett Cannon <brett@python.org> parents: 76406 diff changeset	1442 if dot:
a32be109bd86 Issue #14581: Windows users are allowed to import modules w/o taking Brett Cannon <brett@python.org> parents: 76406 diff changeset	1443 new_name = '{}.{}'.format(name, suffix.lower())
a32be109bd86 Issue #14581: Windows users are allowed to import modules w/o taking Brett Cannon <brett@python.org> parents: 76406 diff changeset	1444 else:
a32be109bd86 Issue #14581: Windows users are allowed to import modules w/o taking Brett Cannon <brett@python.org> parents: 76406 diff changeset	1445 new_name = name
a32be109bd86 Issue #14581: Windows users are allowed to import modules w/o taking Brett Cannon <brett@python.org> parents: 76406 diff changeset	1446 lower_suffix_contents.add(new_name)
a32be109bd86 Issue #14581: Windows users are allowed to import modules w/o taking Brett Cannon <brett@python.org> parents: 76406 diff changeset	1447 self._path_cache = lower_suffix_contents
76753 59b8a7fcb047 Properly mark names in importlib._bootstrap as private. Brett Cannon <brett@python.org> parents: 76586 diff changeset	1448 if sys.platform.startswith(_CASE_INSENSITIVE_PLATFORMS):
82382 2528e4aea338 Issue #17220: Little cleanup of _bootstrap.py. Serhiy Storchaka <storchaka@gmail.com> parents: 82237 diff changeset	1449 self._relaxed_path_cache = {fn.lower() for fn in contents}
62756 34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	1450
76478 1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and Brett Cannon <brett@python.org> parents: 76472 diff changeset	1451 @classmethod
1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and Brett Cannon <brett@python.org> parents: 76472 diff changeset	1452 def path_hook(cls, *loader_details):
1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and Brett Cannon <brett@python.org> parents: 76472 diff changeset	1453 """A class method which returns a closure to use on sys.path_hook
1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and Brett Cannon <brett@python.org> parents: 76472 diff changeset	1454 which will return an instance using the specified loaders and the path
1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and Brett Cannon <brett@python.org> parents: 76472 diff changeset	1455 called on the closure.
52141 f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1456
76478 1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and Brett Cannon <brett@python.org> parents: 76472 diff changeset	1457 If the path called on the closure is not a directory, ImportError is
1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and Brett Cannon <brett@python.org> parents: 76472 diff changeset	1458 raised.
52141 f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1459
76478 1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and Brett Cannon <brett@python.org> parents: 76472 diff changeset	1460 """
1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and Brett Cannon <brett@python.org> parents: 76472 diff changeset	1461 def path_hook_for_FileFinder(path):
1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and Brett Cannon <brett@python.org> parents: 76472 diff changeset	1462 """Path hook for importlib.machinery.FileFinder."""
1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and Brett Cannon <brett@python.org> parents: 76472 diff changeset	1463 if not _path_isdir(path):
1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and Brett Cannon <brett@python.org> parents: 76472 diff changeset	1464 raise ImportError("only directories are supported", path=path)
1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and Brett Cannon <brett@python.org> parents: 76472 diff changeset	1465 return cls(path, *loader_details)
52141 f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1466
76478 1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and Brett Cannon <brett@python.org> parents: 76472 diff changeset	1467 return path_hook_for_FileFinder
52141 f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1468
77600 df856963fc6c A better repr() for FileFinder Antoine Pitrou <solipsis@pitrou.net> parents: 77457 diff changeset	1469 def __repr__(self):
79692 987a8724b220 Remove uses of % with str.format. Brett Cannon <brett@python.org> parents: 79653 diff changeset	1470 return "FileFinder({!r})".format(self.path)
52141 f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1471
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1472
52140 1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap. Brett Cannon <bcannon@gmail.com> parents: 52139 diff changeset	1473 # Import itself ###############################################################
1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap. Brett Cannon <bcannon@gmail.com> parents: 52139 diff changeset	1474
52141 f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1475 class _ImportLockContext:
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1476
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1477 """Context manager for the import lock."""
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1478
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1479 def __enter__(self):
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1480 """Acquire the import lock."""
76331 d777f854a66e Issue #13959: Rename imp to _imp and add Lib/imp.py and begin Brett Cannon <brett@python.org> parents: 76330 diff changeset	1481 _imp.acquire_lock()
52141 f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1482
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1483 def __exit__(self, exc_type, exc_value, exc_traceback):
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1484 """Release the import lock regardless of any raised exceptions."""
76331 d777f854a66e Issue #13959: Rename imp to _imp and add Lib/imp.py and begin Brett Cannon <brett@python.org> parents: 76330 diff changeset	1485 _imp.release_lock()
52141 f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1486
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	1487
74983 03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1488 def _resolve_name(name, package, level):
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1489 """Resolve a relative module name to an absolute one."""
75250 8840037a9c53 unused imports, pep8 Philip Jenvey <pjenvey@underboss.org> parents: 75239 diff changeset	1490 bits = package.rsplit('.', level - 1)
75239 5cfc9c97af23 Simplify importib._resolve_name(). Brett Cannon <brett@python.org> parents: 75235 diff changeset	1491 if len(bits) < level:
5cfc9c97af23 Simplify importib._resolve_name(). Brett Cannon <brett@python.org> parents: 75235 diff changeset	1492 raise ValueError('attempted relative import beyond top-level package')
5cfc9c97af23 Simplify importib._resolve_name(). Brett Cannon <brett@python.org> parents: 75235 diff changeset	1493 base = bits[0]
77964 37850f083af5 Minor refactoring in importlib._bootstrap, and fix the '_wrap' docstring. Florent Xicluna <florent.xicluna@gmail.com> parents: 77917 diff changeset	1494 return '{}.{}'.format(base, name) if name else base
74983 03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1495
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1496
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1497 def _find_module(name, path):
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1498 """Find a module's loader."""
76579 3bd60cc27664 Issue #14605: Stop having implicit entries for sys.meta_path. Brett Cannon <brett@python.org> parents: 76566 diff changeset	1499 if not sys.meta_path:
3bd60cc27664 Issue #14605: Stop having implicit entries for sys.meta_path. Brett Cannon <brett@python.org> parents: 76566 diff changeset	1500 _warnings.warn('sys.meta_path is empty', ImportWarning)
3bd60cc27664 Issue #14605: Stop having implicit entries for sys.meta_path. Brett Cannon <brett@python.org> parents: 76566 diff changeset	1501 for finder in sys.meta_path:
77013 edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1502 with _ImportLockContext():
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1503 loader = finder.find_module(name, path)
74983 03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1504 if loader is not None:
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1505 # The parent import may have already imported this module.
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1506 if name not in sys.modules:
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1507 return loader
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1508 else:
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1509 return sys.modules[name].__loader__
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1510 else:
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1511 return None
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1512
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1513
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1514 def _sanity_check(name, package, level):
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1515 """Verify arguments are "sane"."""
75228 909935a236e3 Do a type check instead of an interface check. Brett Cannon <brett@python.org> parents: 75193 diff changeset	1516 if not isinstance(name, str):
75193 f95faebf5dea Refactor importlib to make it easier to re-implement in C. Brett Cannon <brett@python.org> parents: 75185 diff changeset	1517 raise TypeError("module name must be str, not {}".format(type(name)))
f95faebf5dea Refactor importlib to make it easier to re-implement in C. Brett Cannon <brett@python.org> parents: 75185 diff changeset	1518 if level < 0:
f95faebf5dea Refactor importlib to make it easier to re-implement in C. Brett Cannon <brett@python.org> parents: 75185 diff changeset	1519 raise ValueError('level must be >= 0')
74983 03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1520 if package:
75228 909935a236e3 Do a type check instead of an interface check. Brett Cannon <brett@python.org> parents: 75193 diff changeset	1521 if not isinstance(package, str):
75229 8d1040fdac60 Improper type for __package__ should raise TypeError, not ValueError. Brett Cannon <brett@python.org> parents: 75228 diff changeset	1522 raise TypeError("__package__ not set to a string")
74983 03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1523 elif package not in sys.modules:
77964 37850f083af5 Minor refactoring in importlib._bootstrap, and fix the '_wrap' docstring. Florent Xicluna <florent.xicluna@gmail.com> parents: 77917 diff changeset	1524 msg = ("Parent module {!r} not loaded, cannot perform relative "
74983 03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1525 "import")
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1526 raise SystemError(msg.format(package))
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1527 if not name and level == 0:
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1528 raise ValueError("Empty module name")
52141 f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1529
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	1530
75193 f95faebf5dea Refactor importlib to make it easier to re-implement in C. Brett Cannon <brett@python.org> parents: 75185 diff changeset	1531 _ERR_MSG = 'No module named {!r}'
51639 ef2cba80c975 Change importlib.machinery.PathFinder to not have implicit semantics (that's Brett Cannon <bcannon@gmail.com> parents: 51433 diff changeset	1532
77013 edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1533 def _find_and_load_unlocked(name, import_):
74983 03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1534 path = None
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1535 parent = name.rpartition('.')[0]
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1536 if parent:
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1537 if parent not in sys.modules:
78344 62033490ca0f Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise Nick Coghlan <ncoghlan@gmail.com> parents: 78314 diff changeset	1538 _call_with_frames_removed(import_, parent)
76081 a40cd5976215 If a module injects something into sys.modules as a side-effect of Brett Cannon <brett@python.org> parents: 75463 diff changeset	1539 # Crazy side-effects!
a40cd5976215 If a module injects something into sys.modules as a side-effect of Brett Cannon <brett@python.org> parents: 75463 diff changeset	1540 if name in sys.modules:
a40cd5976215 If a module injects something into sys.modules as a side-effect of Brett Cannon <brett@python.org> parents: 75463 diff changeset	1541 return sys.modules[name]
74983 03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1542 # Backwards-compatibility; be nicer to skip the dict lookup.
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1543 parent_module = sys.modules[parent]
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1544 try:
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1545 path = parent_module.__path__
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1546 except AttributeError:
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1547 msg = (_ERR_MSG + '; {} is not a package').format(name, parent)
76280 c071af95772a Have importlib take advantage of ImportError's new 'name' and 'path' Brett Cannon <brett@python.org> parents: 76081 diff changeset	1548 raise ImportError(msg, name=name)
75193 f95faebf5dea Refactor importlib to make it easier to re-implement in C. Brett Cannon <brett@python.org> parents: 75185 diff changeset	1549 loader = _find_module(name, path)
f95faebf5dea Refactor importlib to make it easier to re-implement in C. Brett Cannon <brett@python.org> parents: 75185 diff changeset	1550 if loader is None:
84103 8a0ed9f63c6e Issue #15767: Introduce ModuleNotFoundError, a subclass of Brett Cannon <brett@python.org> parents: 84091 diff changeset	1551 raise ModuleNotFoundError(_ERR_MSG.format(name), name=name)
76390 db5e3431ee4c rollback 005fd1fe31ab (see #14609 and #14582) Benjamin Peterson <benjamin@python.org> parents: 76384 diff changeset	1552 elif name not in sys.modules:
db5e3431ee4c rollback 005fd1fe31ab (see #14609 and #14582) Benjamin Peterson <benjamin@python.org> parents: 76384 diff changeset	1553 # The parent import may have already imported this module.
db5e3431ee4c rollback 005fd1fe31ab (see #14609 and #14582) Benjamin Peterson <benjamin@python.org> parents: 76384 diff changeset	1554 loader.load_module(name)
76753 59b8a7fcb047 Properly mark names in importlib._bootstrap as private. Brett Cannon <brett@python.org> parents: 76586 diff changeset	1555 _verbose_message('import {!r} # {!r}', name, loader)
76390 db5e3431ee4c rollback 005fd1fe31ab (see #14609 and #14582) Benjamin Peterson <benjamin@python.org> parents: 76384 diff changeset	1556 # Backwards-compatibility; be nicer to skip the dict lookup.
db5e3431ee4c rollback 005fd1fe31ab (see #14609 and #14582) Benjamin Peterson <benjamin@python.org> parents: 76384 diff changeset	1557 module = sys.modules[name]
75193 f95faebf5dea Refactor importlib to make it easier to re-implement in C. Brett Cannon <brett@python.org> parents: 75185 diff changeset	1558 if parent:
f95faebf5dea Refactor importlib to make it easier to re-implement in C. Brett Cannon <brett@python.org> parents: 75185 diff changeset	1559 # Set the module as an attribute on its parent.
f95faebf5dea Refactor importlib to make it easier to re-implement in C. Brett Cannon <brett@python.org> parents: 75185 diff changeset	1560 parent_module = sys.modules[parent]
f95faebf5dea Refactor importlib to make it easier to re-implement in C. Brett Cannon <brett@python.org> parents: 75185 diff changeset	1561 setattr(parent_module, name.rpartition('.')[2], module)
f95faebf5dea Refactor importlib to make it easier to re-implement in C. Brett Cannon <brett@python.org> parents: 75185 diff changeset	1562 # Set __package__ if the loader did not.
77964 37850f083af5 Minor refactoring in importlib._bootstrap, and fix the '_wrap' docstring. Florent Xicluna <florent.xicluna@gmail.com> parents: 77917 diff changeset	1563 if getattr(module, '__package__', None) is None:
75193 f95faebf5dea Refactor importlib to make it easier to re-implement in C. Brett Cannon <brett@python.org> parents: 75185 diff changeset	1564 try:
f95faebf5dea Refactor importlib to make it easier to re-implement in C. Brett Cannon <brett@python.org> parents: 75185 diff changeset	1565 module.__package__ = module.__name__
f95faebf5dea Refactor importlib to make it easier to re-implement in C. Brett Cannon <brett@python.org> parents: 75185 diff changeset	1566 if not hasattr(module, '__path__'):
f95faebf5dea Refactor importlib to make it easier to re-implement in C. Brett Cannon <brett@python.org> parents: 75185 diff changeset	1567 module.__package__ = module.__package__.rpartition('.')[0]
f95faebf5dea Refactor importlib to make it easier to re-implement in C. Brett Cannon <brett@python.org> parents: 75185 diff changeset	1568 except AttributeError:
f95faebf5dea Refactor importlib to make it easier to re-implement in C. Brett Cannon <brett@python.org> parents: 75185 diff changeset	1569 pass
76586 496c68f90a03 Issue #14646: __import__() now sets __loader__ if need be. Brett Cannon <brett@python.org> parents: 76582 diff changeset	1570 # Set loader if need be.
82643 7647aae9481c Issue #17117: Have both import itself and importlib.util.set_loader() Brett Cannon <brett@python.org> parents: 82385 diff changeset	1571 if getattr(module, '__loader__', None) is None:
76586 496c68f90a03 Issue #14646: __import__() now sets __loader__ if need be. Brett Cannon <brett@python.org> parents: 76582 diff changeset	1572 try:
496c68f90a03 Issue #14646: __import__() now sets __loader__ if need be. Brett Cannon <brett@python.org> parents: 76582 diff changeset	1573 module.__loader__ = loader
496c68f90a03 Issue #14646: __import__() now sets __loader__ if need be. Brett Cannon <brett@python.org> parents: 76582 diff changeset	1574 except AttributeError:
496c68f90a03 Issue #14646: __import__() now sets __loader__ if need be. Brett Cannon <brett@python.org> parents: 76582 diff changeset	1575 pass
75193 f95faebf5dea Refactor importlib to make it easier to re-implement in C. Brett Cannon <brett@python.org> parents: 75185 diff changeset	1576 return module
74983 03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1577
66226 af13fb6fb3df Fix a minor inconsistency in capitalization for the 'No module named' exception Brett Cannon <bcannon@gmail.com> parents: 64407 diff changeset	1578
77013 edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1579 def _find_and_load(name, import_):
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1580 """Find and load the module, and release the import lock."""
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1581 try:
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1582 lock = _get_module_lock(name)
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1583 finally:
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1584 _imp.release_lock()
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1585 lock.acquire()
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1586 try:
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1587 return _find_and_load_unlocked(name, import_)
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1588 finally:
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1589 lock.release()
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1590
66226 af13fb6fb3df Fix a minor inconsistency in capitalization for the 'No module named' exception Brett Cannon <bcannon@gmail.com> parents: 64407 diff changeset	1591
51383 a00e5dcea16c Initial, untested stab at writing a common denominator function for __import__ Brett Cannon <bcannon@gmail.com> parents: 51356 diff changeset	1592 def _gcd_import(name, package=None, level=0):
a00e5dcea16c Initial, untested stab at writing a common denominator function for __import__ Brett Cannon <bcannon@gmail.com> parents: 51356 diff changeset	1593 """Import and return the module based on its name, the package the call is
a00e5dcea16c Initial, untested stab at writing a common denominator function for __import__ Brett Cannon <bcannon@gmail.com> parents: 51356 diff changeset	1594 being made from, and the level adjustment.
a00e5dcea16c Initial, untested stab at writing a common denominator function for __import__ Brett Cannon <bcannon@gmail.com> parents: 51356 diff changeset	1595
a00e5dcea16c Initial, untested stab at writing a common denominator function for __import__ Brett Cannon <bcannon@gmail.com> parents: 51356 diff changeset	1596 This function represents the greatest common denominator of functionality
68447 c919a82b5a40 Typos. Eric V. Smith <eric@trueblade.com> parents: 68257 diff changeset	1597 between import_module and __import__. This includes setting __package__ if
51426 7a1356d2aeef Rewrite the code implementing __import__ for importlib. Now it is much simpler Brett Cannon <bcannon@gmail.com> parents: 51383 diff changeset	1598 the loader did not.
7a1356d2aeef Rewrite the code implementing __import__ for importlib. Now it is much simpler Brett Cannon <bcannon@gmail.com> parents: 51383 diff changeset	1599
51383 a00e5dcea16c Initial, untested stab at writing a common denominator function for __import__ Brett Cannon <bcannon@gmail.com> parents: 51356 diff changeset	1600 """
74983 03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1601 _sanity_check(name, package, level)
51383 a00e5dcea16c Initial, untested stab at writing a common denominator function for __import__ Brett Cannon <bcannon@gmail.com> parents: 51356 diff changeset	1602 if level > 0:
74983 03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1603 name = _resolve_name(name, package, level)
77013 edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1604 _imp.acquire_lock()
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1605 if name not in sys.modules:
75193 f95faebf5dea Refactor importlib to make it easier to re-implement in C. Brett Cannon <brett@python.org> parents: 75185 diff changeset	1606 return _find_and_load(name, _gcd_import)
77013 edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1607 module = sys.modules[name]
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1608 if module is None:
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1609 _imp.release_lock()
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1610 message = ("import of {} halted; "
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1611 "None in sys.modules".format(name))
84109 3a50025f1900 Issue #15767: Touch up ModuleNotFoundError usage by import. Brett Cannon <brett@python.org> parents: 84103 diff changeset	1612 raise ModuleNotFoundError(message, name=name)
77013 edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1613 _lock_unlock_module(name)
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1614 return module
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	1615
75235 3670df23081d Turn _return_module() into _handle_fromlist(). Brett Cannon <brett@python.org> parents: 75229 diff changeset	1616 def _handle_fromlist(module, fromlist, import_):
74983 03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1617 """Figure out what __import__ should return.
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	1618
74983 03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1619 The import_ parameter is a callable which takes the name of module to
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1620 import. It is required to decouple the function from assuming importlib's
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1621 import implementation is desired.
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	1622
51426 7a1356d2aeef Rewrite the code implementing __import__ for importlib. Now it is much simpler Brett Cannon <bcannon@gmail.com> parents: 51383 diff changeset	1623 """
7a1356d2aeef Rewrite the code implementing __import__ for importlib. Now it is much simpler Brett Cannon <bcannon@gmail.com> parents: 51383 diff changeset	1624 # The hell that is fromlist ...
75235 3670df23081d Turn _return_module() into _handle_fromlist(). Brett Cannon <brett@python.org> parents: 75229 diff changeset	1625 # If a package was imported, try to import stuff from fromlist.
3670df23081d Turn _return_module() into _handle_fromlist(). Brett Cannon <brett@python.org> parents: 75229 diff changeset	1626 if hasattr(module, '__path__'):
78051 dc18a2a66d16 Issue #15111: When a module was imported using a 'from import' Brett Cannon <brett@python.org> parents: 78042 diff changeset	1627 if '*' in fromlist:
75235 3670df23081d Turn _return_module() into _handle_fromlist(). Brett Cannon <brett@python.org> parents: 75229 diff changeset	1628 fromlist = list(fromlist)
3670df23081d Turn _return_module() into _handle_fromlist(). Brett Cannon <brett@python.org> parents: 75229 diff changeset	1629 fromlist.remove('*')
78051 dc18a2a66d16 Issue #15111: When a module was imported using a 'from import' Brett Cannon <brett@python.org> parents: 78042 diff changeset	1630 if hasattr(module, '__all__'):
dc18a2a66d16 Issue #15111: When a module was imported using a 'from import' Brett Cannon <brett@python.org> parents: 78042 diff changeset	1631 fromlist.extend(module.__all__)
77964 37850f083af5 Minor refactoring in importlib._bootstrap, and fix the '_wrap' docstring. Florent Xicluna <florent.xicluna@gmail.com> parents: 77917 diff changeset	1632 for x in fromlist:
37850f083af5 Minor refactoring in importlib._bootstrap, and fix the '_wrap' docstring. Florent Xicluna <florent.xicluna@gmail.com> parents: 77917 diff changeset	1633 if not hasattr(module, x):
79652 09b5158d5284 Closes issue #15111: Calling __import__ with a module specified in Brett Cannon <brett@python.org> parents: 79228 diff changeset	1634 from_name = '{}.{}'.format(module.__name__, x)
78619 0d52f125dd32 Issue #15715: Ignore failed imports triggered by the use of fromlist. Brett Cannon <brett@python.org> parents: 78511 diff changeset	1635 try:
79652 09b5158d5284 Closes issue #15111: Calling __import__ with a module specified in Brett Cannon <brett@python.org> parents: 79228 diff changeset	1636 _call_with_frames_removed(import_, from_name)
84103 8a0ed9f63c6e Issue #15767: Introduce ModuleNotFoundError, a subclass of Brett Cannon <brett@python.org> parents: 84091 diff changeset	1637 except ModuleNotFoundError as exc:
78619 0d52f125dd32 Issue #15715: Ignore failed imports triggered by the use of fromlist. Brett Cannon <brett@python.org> parents: 78511 diff changeset	1638 # Backwards-compatibility dictates we ignore failed
78758 ca4bf8e10bc0 Issue #15316: Let exceptions raised during imports triggered by the Brett Cannon <brett@python.org> parents: 78738 diff changeset	1639 # imports triggered by fromlist for modules that don't
ca4bf8e10bc0 Issue #15316: Let exceptions raised during imports triggered by the Brett Cannon <brett@python.org> parents: 78738 diff changeset	1640 # exist.
84103 8a0ed9f63c6e Issue #15767: Introduce ModuleNotFoundError, a subclass of Brett Cannon <brett@python.org> parents: 84091 diff changeset	1641 if exc.name == from_name:
8a0ed9f63c6e Issue #15767: Introduce ModuleNotFoundError, a subclass of Brett Cannon <brett@python.org> parents: 84091 diff changeset	1642 continue
79652 09b5158d5284 Closes issue #15111: Calling __import__ with a module specified in Brett Cannon <brett@python.org> parents: 79228 diff changeset	1643 raise
75235 3670df23081d Turn _return_module() into _handle_fromlist(). Brett Cannon <brett@python.org> parents: 75229 diff changeset	1644 return module
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	1645
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	1646
74983 03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1647 def _calc___package__(globals):
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1648 """Calculate what __package__ should be.
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1649
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1650 __package__ is not guaranteed to be defined or could be set to None
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1651 to represent that its proper value is unknown.
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1652
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1653 """
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1654 package = globals.get('__package__')
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1655 if package is None:
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1656 package = globals['__name__']
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1657 if '__path__' not in globals:
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1658 package = package.rpartition('.')[0]
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1659 return package
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1660
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1661
78308 bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	1662 def _get_supported_file_loaders():
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	1663 """Returns a list of file-based module loaders.
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	1664
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	1665 Each item is a tuple (loader, suffixes, allow_packages).
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	1666 """
78486 1db6553f3f8c Issue #15576: Allow extension modules to be a package's __init__ Brett Cannon <brett@python.org> parents: 78485 diff changeset	1667 extensions = ExtensionFileLoader, _imp.extension_suffixes()
1db6553f3f8c Issue #15576: Allow extension modules to be a package's __init__ Brett Cannon <brett@python.org> parents: 78485 diff changeset	1668 source = SourceFileLoader, SOURCE_SUFFIXES
1db6553f3f8c Issue #15576: Allow extension modules to be a package's __init__ Brett Cannon <brett@python.org> parents: 78485 diff changeset	1669 bytecode = SourcelessFileLoader, BYTECODE_SUFFIXES
78308 bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	1670 return [extensions, source, bytecode]
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	1671
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	1672
78456 4240282a9f4a Issue #15471: Don't use mutable object as default values for the Brett Cannon <brett@python.org> parents: 78382 diff changeset	1673 def __import__(name, globals=None, locals=None, fromlist=(), level=0):
51426 7a1356d2aeef Rewrite the code implementing __import__ for importlib. Now it is much simpler Brett Cannon <bcannon@gmail.com> parents: 51383 diff changeset	1674 """Import a module.
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	1675
51426 7a1356d2aeef Rewrite the code implementing __import__ for importlib. Now it is much simpler Brett Cannon <bcannon@gmail.com> parents: 51383 diff changeset	1676 The 'globals' argument is used to infer where the import is occuring from
7a1356d2aeef Rewrite the code implementing __import__ for importlib. Now it is much simpler Brett Cannon <bcannon@gmail.com> parents: 51383 diff changeset	1677 to handle relative imports. The 'locals' argument is ignored. The
7a1356d2aeef Rewrite the code implementing __import__ for importlib. Now it is much simpler Brett Cannon <bcannon@gmail.com> parents: 51383 diff changeset	1678 'fromlist' argument specifies what should exist as attributes on the module
7a1356d2aeef Rewrite the code implementing __import__ for importlib. Now it is much simpler Brett Cannon <bcannon@gmail.com> parents: 51383 diff changeset	1679 being imported (e.g. ``from module import <fromlist>``). The 'level'
7a1356d2aeef Rewrite the code implementing __import__ for importlib. Now it is much simpler Brett Cannon <bcannon@gmail.com> parents: 51383 diff changeset	1680 argument represents the package location to import from in a relative
7a1356d2aeef Rewrite the code implementing __import__ for importlib. Now it is much simpler Brett Cannon <bcannon@gmail.com> parents: 51383 diff changeset	1681 import (e.g. ``from ..pkg import mod`` would have a 'level' of 2).
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	1682
51426 7a1356d2aeef Rewrite the code implementing __import__ for importlib. Now it is much simpler Brett Cannon <bcannon@gmail.com> parents: 51383 diff changeset	1683 """
7a1356d2aeef Rewrite the code implementing __import__ for importlib. Now it is much simpler Brett Cannon <bcannon@gmail.com> parents: 51383 diff changeset	1684 if level == 0:
7a1356d2aeef Rewrite the code implementing __import__ for importlib. Now it is much simpler Brett Cannon <bcannon@gmail.com> parents: 51383 diff changeset	1685 module = _gcd_import(name)
7a1356d2aeef Rewrite the code implementing __import__ for importlib. Now it is much simpler Brett Cannon <bcannon@gmail.com> parents: 51383 diff changeset	1686 else:
78456 4240282a9f4a Issue #15471: Don't use mutable object as default values for the Brett Cannon <brett@python.org> parents: 78382 diff changeset	1687 globals_ = globals if globals is not None else {}
4240282a9f4a Issue #15471: Don't use mutable object as default values for the Brett Cannon <brett@python.org> parents: 78382 diff changeset	1688 package = _calc___package__(globals_)
51426 7a1356d2aeef Rewrite the code implementing __import__ for importlib. Now it is much simpler Brett Cannon <bcannon@gmail.com> parents: 51383 diff changeset	1689 module = _gcd_import(name, package, level)
7a1356d2aeef Rewrite the code implementing __import__ for importlib. Now it is much simpler Brett Cannon <bcannon@gmail.com> parents: 51383 diff changeset	1690 if not fromlist:
7a1356d2aeef Rewrite the code implementing __import__ for importlib. Now it is much simpler Brett Cannon <bcannon@gmail.com> parents: 51383 diff changeset	1691 # Return up to the first dot in 'name'. This is complicated by the fact
7a1356d2aeef Rewrite the code implementing __import__ for importlib. Now it is much simpler Brett Cannon <bcannon@gmail.com> parents: 51383 diff changeset	1692 # that 'name' may be relative.
7a1356d2aeef Rewrite the code implementing __import__ for importlib. Now it is much simpler Brett Cannon <bcannon@gmail.com> parents: 51383 diff changeset	1693 if level == 0:
76826 d6324941b739 Issue #14583: Fix importlib bug when a package's __init__.py would first import one of its modules then raise an error. Antoine Pitrou <solipsis@pitrou.net> parents: 76765 diff changeset	1694 return _gcd_import(name.partition('.')[0])
51426 7a1356d2aeef Rewrite the code implementing __import__ for importlib. Now it is much simpler Brett Cannon <bcannon@gmail.com> parents: 51383 diff changeset	1695 elif not name:
7a1356d2aeef Rewrite the code implementing __import__ for importlib. Now it is much simpler Brett Cannon <bcannon@gmail.com> parents: 51383 diff changeset	1696 return module
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	1697 else:
79228 4a979326e6d6 Add some comments. Brett Cannon <brett@python.org> parents: 79191 diff changeset	1698 # Figure out where to slice the module's name up to the first dot
4a979326e6d6 Add some comments. Brett Cannon <brett@python.org> parents: 79191 diff changeset	1699 # in 'name'.
51426 7a1356d2aeef Rewrite the code implementing __import__ for importlib. Now it is much simpler Brett Cannon <bcannon@gmail.com> parents: 51383 diff changeset	1700 cut_off = len(name) - len(name.partition('.')[0])
79228 4a979326e6d6 Add some comments. Brett Cannon <brett@python.org> parents: 79191 diff changeset	1701 # Slice end needs to be positive to alleviate need to special-case
4a979326e6d6 Add some comments. Brett Cannon <brett@python.org> parents: 79191 diff changeset	1702 # when ``'.' not in name``.
76312 9e8cbf07068a Handle importing pkg.mod by executing Brett Cannon <brett@python.org> parents: 76309 diff changeset	1703 return sys.modules[module.__name__[:len(module.__name__)-cut_off]]
75235 3670df23081d Turn _return_module() into _handle_fromlist(). Brett Cannon <brett@python.org> parents: 75229 diff changeset	1704 else:
3670df23081d Turn _return_module() into _handle_fromlist(). Brett Cannon <brett@python.org> parents: 75229 diff changeset	1705 return _handle_fromlist(module, fromlist, _gcd_import)
74983 03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1706
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1707
76445 c820aa9c0c00 Issue #13959: Keep imp.get_magic() in C code, but cache in importlib Brett Cannon <brett@python.org> parents: 76428 diff changeset	1708
76331 d777f854a66e Issue #13959: Rename imp to _imp and add Lib/imp.py and begin Brett Cannon <brett@python.org> parents: 76330 diff changeset	1709 def _setup(sys_module, _imp_module):
74839 322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1710 """Setup importlib by importing needed built-in modules and injecting them
322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1711 into the global namespace.
322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1712
76331 d777f854a66e Issue #13959: Rename imp to _imp and add Lib/imp.py and begin Brett Cannon <brett@python.org> parents: 76330 diff changeset	1713 As sys is needed for sys.modules access and _imp is needed to load built-in
74935 81f3bb5efa40 Bring importlib in line w/ changes made in my personal bootstrap branch in the sandbox. Brett Cannon <brett@python.org> parents: 74867 diff changeset	1714 modules, those two modules must be explicitly passed in.
74839 322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1715
322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1716 """
79173 4de5e4ec3cff don't depend on __debug__ because it's baked in at freeze time (issue #16046) Benjamin Peterson <benjamin@python.org> parents: 78779 diff changeset	1717 global _imp, sys, BYTECODE_SUFFIXES
76331 d777f854a66e Issue #13959: Rename imp to _imp and add Lib/imp.py and begin Brett Cannon <brett@python.org> parents: 76330 diff changeset	1718 _imp = _imp_module
74839 322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1719 sys = sys_module
322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1720
79173 4de5e4ec3cff don't depend on __debug__ because it's baked in at freeze time (issue #16046) Benjamin Peterson <benjamin@python.org> parents: 78779 diff changeset	1721 if sys.flags.optimize:
4de5e4ec3cff don't depend on __debug__ because it's baked in at freeze time (issue #16046) Benjamin Peterson <benjamin@python.org> parents: 78779 diff changeset	1722 BYTECODE_SUFFIXES = OPTIMIZED_BYTECODE_SUFFIXES
51426 7a1356d2aeef Rewrite the code implementing __import__ for importlib. Now it is much simpler Brett Cannon <bcannon@gmail.com> parents: 51383 diff changeset	1723 else:
79173 4de5e4ec3cff don't depend on __debug__ because it's baked in at freeze time (issue #16046) Benjamin Peterson <benjamin@python.org> parents: 78779 diff changeset	1724 BYTECODE_SUFFIXES = DEBUG_BYTECODE_SUFFIXES
4de5e4ec3cff don't depend on __debug__ because it's baked in at freeze time (issue #16046) Benjamin Peterson <benjamin@python.org> parents: 78779 diff changeset	1725
81869 05747d3bcd9c Issue #17098: Make sure every module has __loader__ defined. Brett Cannon <brett@python.org> parents: 81427 diff changeset	1726 module_type = type(sys)
81884 19ea454ccdf7 Issue #17098: Be more stringent of setting __loader__ on early imported Brett Cannon <brett@python.org> parents: 81869 diff changeset	1727 for name, module in sys.modules.items():
81869 05747d3bcd9c Issue #17098: Make sure every module has __loader__ defined. Brett Cannon <brett@python.org> parents: 81427 diff changeset	1728 if isinstance(module, module_type):
83607 e39a8f8ceb9f #17115,17116: Have modules initialize the __package__ and __loader__ Brett Cannon <brett@python.org> parents: 83572 diff changeset	1729 if getattr(module, '__loader__', None) is None:
81884 19ea454ccdf7 Issue #17098: Be more stringent of setting __loader__ on early imported Brett Cannon <brett@python.org> parents: 81869 diff changeset	1730 if name in sys.builtin_module_names:
19ea454ccdf7 Issue #17098: Be more stringent of setting __loader__ on early imported Brett Cannon <brett@python.org> parents: 81869 diff changeset	1731 module.__loader__ = BuiltinImporter
19ea454ccdf7 Issue #17098: Be more stringent of setting __loader__ on early imported Brett Cannon <brett@python.org> parents: 81869 diff changeset	1732 elif _imp.is_frozen(name):
19ea454ccdf7 Issue #17098: Be more stringent of setting __loader__ on early imported Brett Cannon <brett@python.org> parents: 81869 diff changeset	1733 module.__loader__ = FrozenImporter
74839 322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1734
322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1735 self_module = sys.modules[__name__]
322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1736 for builtin_name in ('_io', '_warnings', 'builtins', 'marshal'):
322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1737 if builtin_name not in sys.modules:
322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1738 builtin_module = BuiltinImporter.load_module(builtin_name)
322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1739 else:
322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1740 builtin_module = sys.modules[builtin_name]
322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1741 setattr(self_module, builtin_name, builtin_module)
322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1742
79480 15949f34e08f #16135: Removal of OS/2 support (Python code partial cleanup) Jesus Cea <jcea@jcea.es> parents: 79228 diff changeset	1743 os_details = ('posix', ['/']), ('nt', ['\\', '/'])
76449 163360370ad2 Make path manipulation more robust for platforms with alternative path Brett Cannon <brett@python.org> parents: 76445 diff changeset	1744 for builtin_os, path_separators in os_details:
76472 4e853913054c Issue #13959: Continue to try to accomodate altsep in importlib by not Brett Cannon <brett@python.org> parents: 76466 diff changeset	1745 # Assumption made in _path_join()
4e853913054c Issue #13959: Continue to try to accomodate altsep in importlib by not Brett Cannon <brett@python.org> parents: 76466 diff changeset	1746 assert all(len(sep) == 1 for sep in path_separators)
76449 163360370ad2 Make path manipulation more robust for platforms with alternative path Brett Cannon <brett@python.org> parents: 76445 diff changeset	1747 path_sep = path_separators[0]
74839 322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1748 if builtin_os in sys.modules:
322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1749 os_module = sys.modules[builtin_os]
322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1750 break
322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1751 else:
322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1752 try:
322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1753 os_module = BuiltinImporter.load_module(builtin_os)
322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1754 break
322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1755 except ImportError:
322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1756 continue
322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1757 else:
322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1758 raise ImportError('importlib requires posix or nt')
77013 edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1759
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1760 try:
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1761 thread_module = BuiltinImporter.load_module('_thread')
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1762 except ImportError:
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1763 # Python was built without threads
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1764 thread_module = None
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1765 weakref_module = BuiltinImporter.load_module('_weakref')
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1766
78308 bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	1767 if builtin_os == 'nt':
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	1768 winreg_module = BuiltinImporter.load_module('winreg')
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	1769 setattr(self_module, '_winreg', winreg_module)
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	1770
74839 322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1771 setattr(self_module, '_os', os_module)
77013 edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1772 setattr(self_module, '_thread', thread_module)
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1773 setattr(self_module, '_weakref', weakref_module)
74839 322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1774 setattr(self_module, 'path_sep', path_sep)
82382 2528e4aea338 Issue #17220: Little cleanup of _bootstrap.py. Serhiy Storchaka <storchaka@gmail.com> parents: 82237 diff changeset	1775 setattr(self_module, 'path_separators', ''.join(path_separators))
75048 bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function. Antoine Pitrou <solipsis@pitrou.net> parents: 75045 diff changeset	1776 # Constants
75055 b8576719fe71 _relax_case -> _make_relax_case Antoine Pitrou <solipsis@pitrou.net> parents: 75048 diff changeset	1777 setattr(self_module, '_relax_case', _make_relax_case())
78486 1db6553f3f8c Issue #15576: Allow extension modules to be a package's __init__ Brett Cannon <brett@python.org> parents: 78485 diff changeset	1778 EXTENSION_SUFFIXES.extend(_imp.extension_suffixes())
76451 ea46ebba8a0f Issue #13959: Re-implement imp.source_from_cache() in Lib/imp.py. Brett Cannon <brett@python.org> parents: 76450 diff changeset	1779 if builtin_os == 'nt':
76874 b81ddaf0db47 Issue #13959: Deprecate imp.get_suffixes() for new attributes on Brett Cannon <brett@python.org> parents: 76826 diff changeset	1780 SOURCE_SUFFIXES.append('.pyw')
78486 1db6553f3f8c Issue #15576: Allow extension modules to be a package's __init__ Brett Cannon <brett@python.org> parents: 78485 diff changeset	1781 if '_d.pyd' in EXTENSION_SUFFIXES:
78380 a1ac1e13c5a0 Close #15519: Properly expose WindowsRegistryFinder in importlib and bring the name into line with normal import terminology. Original patch by Eric Snow Nick Coghlan <ncoghlan@gmail.com> parents: 78344 diff changeset	1782 WindowsRegistryFinder.DEBUG_BUILD = True
75000 e477832212b9 Optimize importlib's case-sensitivity check by wasting as little time as possible under case-sensitive OSs. Brett Cannon <brett@python.org> parents: 74998 diff changeset	1783
74935 81f3bb5efa40 Bring importlib in line w/ changes made in my personal bootstrap branch in the sandbox. Brett Cannon <brett@python.org> parents: 74867 diff changeset	1784
76331 d777f854a66e Issue #13959: Rename imp to _imp and add Lib/imp.py and begin Brett Cannon <brett@python.org> parents: 76330 diff changeset	1785 def _install(sys_module, _imp_module):
76765 146001e3364d Clean up a docstring. Brett Cannon <brett@python.org> parents: 76755 diff changeset	1786 """Install importlib as the implementation of import."""
76331 d777f854a66e Issue #13959: Rename imp to _imp and add Lib/imp.py and begin Brett Cannon <brett@python.org> parents: 76330 diff changeset	1787 _setup(sys_module, _imp_module)
78308 bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	1788 supported_loaders = _get_supported_file_loaders()
76582 7025ee00dbf6 Issue #14605: Use None in sys.path_importer_cache to represent no Brett Cannon <brett@python.org> parents: 76579 diff changeset	1789 sys.path_hooks.extend([FileFinder.path_hook(*supported_loaders)])
78308 bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	1790 sys.meta_path.append(BuiltinImporter)
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	1791 sys.meta_path.append(FrozenImporter)
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	1792 if _os.__name__ == 'nt':
78380 a1ac1e13c5a0 Close #15519: Properly expose WindowsRegistryFinder in importlib and bring the name into line with normal import terminology. Original patch by Eric Snow Nick Coghlan <ncoghlan@gmail.com> parents: 78344 diff changeset	1793 sys.meta_path.append(WindowsRegistryFinder)
78308 bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	1794 sys.meta_path.append(PathFinder)

Mercurial > cpython-lru_cache-weakref

annotate Lib/importlib/_bootstrap.py @ 84172:e353f64dfd95