cpython-lru_cache-weakref: Lib/importlib/

annotate Lib/importlib/_bootstrap.py @ 78489:e7a67f1bf604

Issue #15502: Refactor some code.

author	Brett Cannon <brett@python.org>
date	Fri, 10 Aug 2012 17:41:23 -0400
parents	e024f6ba5ed8
children	f7b59e890e30

rev	line source
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	1 """Core implementation of import.
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	2
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	3 This module is NOT meant to be directly imported! It has been designed such
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	4 that it can be bootstrapped into Python as the implementation of import. As
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	5 such it requires the injection of specific modules and attributes in order to
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	6 work. One should use importlib as the public-facing version of this module.
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	7
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	8 """
76549 acfdf46b8de1 Issue #14605 and #14642: Marc-Andre Lemburg <mal@egenix.com> parents: 76548 diff changeset	9 #
acfdf46b8de1 Issue #14605 and #14642: Marc-Andre Lemburg <mal@egenix.com> parents: 76548 diff changeset	10 # IMPORTANT: Whenever making changes to this module, be sure to run
acfdf46b8de1 Issue #14605 and #14642: Marc-Andre Lemburg <mal@egenix.com> parents: 76548 diff changeset	11 # a top-level make in order to get the frozen version of the module
acfdf46b8de1 Issue #14605 and #14642: Marc-Andre Lemburg <mal@egenix.com> parents: 76548 diff changeset	12 # update. Not doing so, will result in the Makefile to fail for
acfdf46b8de1 Issue #14605 and #14642: Marc-Andre Lemburg <mal@egenix.com> parents: 76548 diff changeset	13 # all others who don't have a ./python around to freeze the module
acfdf46b8de1 Issue #14605 and #14642: Marc-Andre Lemburg <mal@egenix.com> parents: 76548 diff changeset	14 # in the early stages of compilation.
acfdf46b8de1 Issue #14605 and #14642: Marc-Andre Lemburg <mal@egenix.com> parents: 76548 diff changeset	15 #
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	16
76449 163360370ad2 Make path manipulation more robust for platforms with alternative path Brett Cannon <brett@python.org> parents: 76445 diff changeset	17 # See importlib._setup() for what is injected into the global namespace.
163360370ad2 Make path manipulation more robust for platforms with alternative path Brett Cannon <brett@python.org> parents: 76445 diff changeset	18
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	19 # When editing this code be aware that code executed at import time CANNOT
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	20 # reference any injected objects! This includes not only global code but also
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	21 # anything specified at the class level.
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	22
76478 1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and Brett Cannon <brett@python.org> parents: 76472 diff changeset	23 # XXX Make sure all public names have no single leading underscore and all
1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and Brett Cannon <brett@python.org> parents: 76472 diff changeset	24 # others do.
1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and Brett Cannon <brett@python.org> parents: 76472 diff changeset	25
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	26
52140 1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap. Brett Cannon <bcannon@gmail.com> parents: 52139 diff changeset	27 # Bootstrap-related code ######################################################
1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap. Brett Cannon <bcannon@gmail.com> parents: 52139 diff changeset	28
76753 59b8a7fcb047 Properly mark names in importlib._bootstrap as private. Brett Cannon <brett@python.org> parents: 76586 diff changeset	29 _CASE_INSENSITIVE_PLATFORMS = 'win', 'cygwin', 'darwin'
74643 ecf4a7bb8807 Relocate importlib._case_ok to importlib._bootstrap. Brett Cannon <brett@python.org> parents: 74630 diff changeset	30
ecf4a7bb8807 Relocate importlib._case_ok to importlib._bootstrap. Brett Cannon <brett@python.org> parents: 74630 diff changeset	31
75055 b8576719fe71 _relax_case -> _make_relax_case Antoine Pitrou <solipsis@pitrou.net> parents: 75048 diff changeset	32 def _make_relax_case():
76753 59b8a7fcb047 Properly mark names in importlib._bootstrap as private. Brett Cannon <brett@python.org> parents: 76586 diff changeset	33 if sys.platform.startswith(_CASE_INSENSITIVE_PLATFORMS):
75048 bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function. Antoine Pitrou <solipsis@pitrou.net> parents: 75045 diff changeset	34 def _relax_case():
75070 defd944af91b put docstrings on functions Benjamin Peterson <benjamin@python.org> parents: 75055 diff changeset	35 """True if filenames must be checked case-insensitively."""
75048 bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function. Antoine Pitrou <solipsis@pitrou.net> parents: 75045 diff changeset	36 return b'PYTHONCASEOK' in _os.environ
74643 ecf4a7bb8807 Relocate importlib._case_ok to importlib._bootstrap. Brett Cannon <brett@python.org> parents: 74630 diff changeset	37 else:
75048 bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function. Antoine Pitrou <solipsis@pitrou.net> parents: 75045 diff changeset	38 def _relax_case():
75070 defd944af91b put docstrings on functions Benjamin Peterson <benjamin@python.org> parents: 75055 diff changeset	39 """True if filenames must be checked case-insensitively."""
75048 bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function. Antoine Pitrou <solipsis@pitrou.net> parents: 75045 diff changeset	40 return False
bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function. Antoine Pitrou <solipsis@pitrou.net> parents: 75045 diff changeset	41 return _relax_case
74643 ecf4a7bb8807 Relocate importlib._case_ok to importlib._bootstrap. Brett Cannon <brett@python.org> parents: 74630 diff changeset	42
ecf4a7bb8807 Relocate importlib._case_ok to importlib._bootstrap. Brett Cannon <brett@python.org> parents: 74630 diff changeset	43
74630 7d8f016784cd Move some code from importlib.__init__ to importlib._bootstrap that Brett Cannon <brett@python.org> parents: 74449 diff changeset	44 # TODO: Expose from marshal
7d8f016784cd Move some code from importlib.__init__ to importlib._bootstrap that Brett Cannon <brett@python.org> parents: 74449 diff changeset	45 def _w_long(x):
7d8f016784cd Move some code from importlib.__init__ to importlib._bootstrap that Brett Cannon <brett@python.org> parents: 74449 diff changeset	46 """Convert a 32-bit integer to little-endian.
7d8f016784cd Move some code from importlib.__init__ to importlib._bootstrap that Brett Cannon <brett@python.org> parents: 74449 diff changeset	47
7d8f016784cd Move some code from importlib.__init__ to importlib._bootstrap that Brett Cannon <brett@python.org> parents: 74449 diff changeset	48 XXX Temporary until marshal's long functions are exposed.
7d8f016784cd Move some code from importlib.__init__ to importlib._bootstrap that Brett Cannon <brett@python.org> parents: 74449 diff changeset	49
7d8f016784cd Move some code from importlib.__init__ to importlib._bootstrap that Brett Cannon <brett@python.org> parents: 74449 diff changeset	50 """
7d8f016784cd Move some code from importlib.__init__ to importlib._bootstrap that Brett Cannon <brett@python.org> parents: 74449 diff changeset	51 x = int(x)
7d8f016784cd Move some code from importlib.__init__ to importlib._bootstrap that Brett Cannon <brett@python.org> parents: 74449 diff changeset	52 int_bytes = []
7d8f016784cd Move some code from importlib.__init__ to importlib._bootstrap that Brett Cannon <brett@python.org> parents: 74449 diff changeset	53 int_bytes.append(x & 0xFF)
7d8f016784cd Move some code from importlib.__init__ to importlib._bootstrap that Brett Cannon <brett@python.org> parents: 74449 diff changeset	54 int_bytes.append((x >> 8) & 0xFF)
7d8f016784cd Move some code from importlib.__init__ to importlib._bootstrap that Brett Cannon <brett@python.org> parents: 74449 diff changeset	55 int_bytes.append((x >> 16) & 0xFF)
7d8f016784cd Move some code from importlib.__init__ to importlib._bootstrap that Brett Cannon <brett@python.org> parents: 74449 diff changeset	56 int_bytes.append((x >> 24) & 0xFF)
7d8f016784cd Move some code from importlib.__init__ to importlib._bootstrap that Brett Cannon <brett@python.org> parents: 74449 diff changeset	57 return bytearray(int_bytes)
7d8f016784cd Move some code from importlib.__init__ to importlib._bootstrap that Brett Cannon <brett@python.org> parents: 74449 diff changeset	58
7d8f016784cd Move some code from importlib.__init__ to importlib._bootstrap that Brett Cannon <brett@python.org> parents: 74449 diff changeset	59
7d8f016784cd Move some code from importlib.__init__ to importlib._bootstrap that Brett Cannon <brett@python.org> parents: 74449 diff changeset	60 # TODO: Expose from marshal
7d8f016784cd Move some code from importlib.__init__ to importlib._bootstrap that Brett Cannon <brett@python.org> parents: 74449 diff changeset	61 def _r_long(int_bytes):
7d8f016784cd Move some code from importlib.__init__ to importlib._bootstrap that Brett Cannon <brett@python.org> parents: 74449 diff changeset	62 """Convert 4 bytes in little-endian to an integer.
7d8f016784cd Move some code from importlib.__init__ to importlib._bootstrap that Brett Cannon <brett@python.org> parents: 74449 diff changeset	63
7d8f016784cd Move some code from importlib.__init__ to importlib._bootstrap that Brett Cannon <brett@python.org> parents: 74449 diff changeset	64 XXX Temporary until marshal's long function are exposed.
7d8f016784cd Move some code from importlib.__init__ to importlib._bootstrap that Brett Cannon <brett@python.org> parents: 74449 diff changeset	65
7d8f016784cd Move some code from importlib.__init__ to importlib._bootstrap that Brett Cannon <brett@python.org> parents: 74449 diff changeset	66 """
7d8f016784cd Move some code from importlib.__init__ to importlib._bootstrap that Brett Cannon <brett@python.org> parents: 74449 diff changeset	67 x = int_bytes[0]
7d8f016784cd Move some code from importlib.__init__ to importlib._bootstrap that Brett Cannon <brett@python.org> parents: 74449 diff changeset	68 x \|= int_bytes[1] << 8
7d8f016784cd Move some code from importlib.__init__ to importlib._bootstrap that Brett Cannon <brett@python.org> parents: 74449 diff changeset	69 x \|= int_bytes[2] << 16
7d8f016784cd Move some code from importlib.__init__ to importlib._bootstrap that Brett Cannon <brett@python.org> parents: 74449 diff changeset	70 x \|= int_bytes[3] << 24
7d8f016784cd Move some code from importlib.__init__ to importlib._bootstrap that Brett Cannon <brett@python.org> parents: 74449 diff changeset	71 return x
7d8f016784cd Move some code from importlib.__init__ to importlib._bootstrap that Brett Cannon <brett@python.org> parents: 74449 diff changeset	72
7d8f016784cd Move some code from importlib.__init__ to importlib._bootstrap that Brett Cannon <brett@python.org> parents: 74449 diff changeset	73
76472 4e853913054c Issue #13959: Continue to try to accomodate altsep in importlib by not Brett Cannon <brett@python.org> parents: 76466 diff changeset	74 def _path_join(*path_parts):
76449 163360370ad2 Make path manipulation more robust for platforms with alternative path Brett Cannon <brett@python.org> parents: 76445 diff changeset	75 """Replacement for os.path.join()."""
76472 4e853913054c Issue #13959: Continue to try to accomodate altsep in importlib by not Brett Cannon <brett@python.org> parents: 76466 diff changeset	76 new_parts = []
4e853913054c Issue #13959: Continue to try to accomodate altsep in importlib by not Brett Cannon <brett@python.org> parents: 76466 diff changeset	77 for part in path_parts:
4e853913054c Issue #13959: Continue to try to accomodate altsep in importlib by not Brett Cannon <brett@python.org> parents: 76466 diff changeset	78 if not part:
4e853913054c Issue #13959: Continue to try to accomodate altsep in importlib by not Brett Cannon <brett@python.org> parents: 76466 diff changeset	79 continue
4e853913054c Issue #13959: Continue to try to accomodate altsep in importlib by not Brett Cannon <brett@python.org> parents: 76466 diff changeset	80 new_parts.append(part)
4e853913054c Issue #13959: Continue to try to accomodate altsep in importlib by not Brett Cannon <brett@python.org> parents: 76466 diff changeset	81 if part[-1] not in path_separators:
4e853913054c Issue #13959: Continue to try to accomodate altsep in importlib by not Brett Cannon <brett@python.org> parents: 76466 diff changeset	82 new_parts.append(path_sep)
4e853913054c Issue #13959: Continue to try to accomodate altsep in importlib by not Brett Cannon <brett@python.org> parents: 76466 diff changeset	83 return ''.join(new_parts[:-1]) # Drop superfluous path separator.
74630 7d8f016784cd Move some code from importlib.__init__ to importlib._bootstrap that Brett Cannon <brett@python.org> parents: 74449 diff changeset	84
76449 163360370ad2 Make path manipulation more robust for platforms with alternative path Brett Cannon <brett@python.org> parents: 76445 diff changeset	85
163360370ad2 Make path manipulation more robust for platforms with alternative path Brett Cannon <brett@python.org> parents: 76445 diff changeset	86 def _path_split(path):
163360370ad2 Make path manipulation more robust for platforms with alternative path Brett Cannon <brett@python.org> parents: 76445 diff changeset	87 """Replacement for os.path.split()."""
163360370ad2 Make path manipulation more robust for platforms with alternative path Brett Cannon <brett@python.org> parents: 76445 diff changeset	88 for x in reversed(path):
163360370ad2 Make path manipulation more robust for platforms with alternative path Brett Cannon <brett@python.org> parents: 76445 diff changeset	89 if x in path_separators:
163360370ad2 Make path manipulation more robust for platforms with alternative path Brett Cannon <brett@python.org> parents: 76445 diff changeset	90 sep = x
163360370ad2 Make path manipulation more robust for platforms with alternative path Brett Cannon <brett@python.org> parents: 76445 diff changeset	91 break
163360370ad2 Make path manipulation more robust for platforms with alternative path Brett Cannon <brett@python.org> parents: 76445 diff changeset	92 else:
163360370ad2 Make path manipulation more robust for platforms with alternative path Brett Cannon <brett@python.org> parents: 76445 diff changeset	93 sep = path_sep
163360370ad2 Make path manipulation more robust for platforms with alternative path Brett Cannon <brett@python.org> parents: 76445 diff changeset	94 front, _, tail = path.rpartition(sep)
163360370ad2 Make path manipulation more robust for platforms with alternative path Brett Cannon <brett@python.org> parents: 76445 diff changeset	95 return front, tail
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	96
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	97
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	98 def _path_is_mode_type(path, mode):
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	99 """Test whether the path is the specified mode type."""
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	100 try:
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	101 stat_info = _os.stat(path)
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	102 except OSError:
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	103 return False
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	104 return (stat_info.st_mode & 0o170000) == mode
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	105
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	106
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	107 # XXX Could also expose Modules/getpath.c:isfile()
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	108 def _path_isfile(path):
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	109 """Replacement for os.path.isfile."""
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	110 return _path_is_mode_type(path, 0o100000)
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	111
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	112
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	113 # XXX Could also expose Modules/getpath.c:isdir()
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	114 def _path_isdir(path):
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	115 """Replacement for os.path.isdir."""
62756 34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	116 if not path:
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	117 path = _os.getcwd()
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	118 return _path_is_mode_type(path, 0o040000)
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	119
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	120
72960 c16063765d3a Issue #13146: Writing a pyc file is now atomic under POSIX. Antoine Pitrou <solipsis@pitrou.net> parents: 68887 diff changeset	121 def _write_atomic(path, data):
75185 27d31f0c4ad5 Issue #14077: importlib: Fix regression introduced by de6703671386. Charles-François Natali <neologix@free.fr> parents: 75082 diff changeset	122 """Best-effort function to write data to a path atomically.
27d31f0c4ad5 Issue #14077: importlib: Fix regression introduced by de6703671386. Charles-François Natali <neologix@free.fr> parents: 75082 diff changeset	123 Be prepared to handle a FileExistsError if concurrent writing of the
27d31f0c4ad5 Issue #14077: importlib: Fix regression introduced by de6703671386. Charles-François Natali <neologix@free.fr> parents: 75082 diff changeset	124 temporary file is attempted."""
73573 b75b41237380 Issue #13392: Writing a pyc file should now be atomic under Windows as well. Antoine Pitrou <solipsis@pitrou.net> parents: 73477 diff changeset	125 # id() is used to generate a pseudo-random filename.
b75b41237380 Issue #13392: Writing a pyc file should now be atomic under Windows as well. Antoine Pitrou <solipsis@pitrou.net> parents: 73477 diff changeset	126 path_tmp = '{}.{}'.format(path, id(path))
b75b41237380 Issue #13392: Writing a pyc file should now be atomic under Windows as well. Antoine Pitrou <solipsis@pitrou.net> parents: 73477 diff changeset	127 fd = _os.open(path_tmp, _os.O_EXCL \| _os.O_CREAT \| _os.O_WRONLY, 0o666)
b75b41237380 Issue #13392: Writing a pyc file should now be atomic under Windows as well. Antoine Pitrou <solipsis@pitrou.net> parents: 73477 diff changeset	128 try:
74998 de6703671386 Have importlib use os.replace() for atomic renaming. Brett Cannon <brett@python.org> parents: 74994 diff changeset	129 # We first write data to a temporary file, and then use os.replace() to
de6703671386 Have importlib use os.replace() for atomic renaming. Brett Cannon <brett@python.org> parents: 74994 diff changeset	130 # perform an atomic rename.
73573 b75b41237380 Issue #13392: Writing a pyc file should now be atomic under Windows as well. Antoine Pitrou <solipsis@pitrou.net> parents: 73477 diff changeset	131 with _io.FileIO(fd, 'wb') as file:
b75b41237380 Issue #13392: Writing a pyc file should now be atomic under Windows as well. Antoine Pitrou <solipsis@pitrou.net> parents: 73477 diff changeset	132 file.write(data)
74998 de6703671386 Have importlib use os.replace() for atomic renaming. Brett Cannon <brett@python.org> parents: 74994 diff changeset	133 _os.replace(path_tmp, path)
73573 b75b41237380 Issue #13392: Writing a pyc file should now be atomic under Windows as well. Antoine Pitrou <solipsis@pitrou.net> parents: 73477 diff changeset	134 except OSError:
b75b41237380 Issue #13392: Writing a pyc file should now be atomic under Windows as well. Antoine Pitrou <solipsis@pitrou.net> parents: 73477 diff changeset	135 try:
b75b41237380 Issue #13392: Writing a pyc file should now be atomic under Windows as well. Antoine Pitrou <solipsis@pitrou.net> parents: 73477 diff changeset	136 _os.unlink(path_tmp)
72960 c16063765d3a Issue #13146: Writing a pyc file is now atomic under POSIX. Antoine Pitrou <solipsis@pitrou.net> parents: 68887 diff changeset	137 except OSError:
73573 b75b41237380 Issue #13392: Writing a pyc file should now be atomic under Windows as well. Antoine Pitrou <solipsis@pitrou.net> parents: 73477 diff changeset	138 pass
b75b41237380 Issue #13392: Writing a pyc file should now be atomic under Windows as well. Antoine Pitrou <solipsis@pitrou.net> parents: 73477 diff changeset	139 raise
72960 c16063765d3a Issue #13146: Writing a pyc file is now atomic under POSIX. Antoine Pitrou <solipsis@pitrou.net> parents: 68887 diff changeset	140
c16063765d3a Issue #13146: Writing a pyc file is now atomic under POSIX. Antoine Pitrou <solipsis@pitrou.net> parents: 68887 diff changeset	141
52139 2e8e897a1244 Make utility code in importlib._bootstrap private. Brett Cannon <bcannon@gmail.com> parents: 52115 diff changeset	142 def _wrap(new, old):
77964 37850f083af5 Minor refactoring in importlib._bootstrap, and fix the '_wrap' docstring. Florent Xicluna <florent.xicluna@gmail.com> parents: 77917 diff changeset	143 """Simple substitute for functools.update_wrapper."""
73977 54a77c556d9a Issue #13593: updating the importlib utility decorators for __qualname__. Meador Inge <meadori@gmail.com> parents: 73974 diff changeset	144 for replace in ['__module__', '__name__', '__qualname__', '__doc__']:
74838 7a6fd7cd16e9 Don't fail in the face of a lacking attribute when wrapping a Brett Cannon <brett@python.org> parents: 74643 diff changeset	145 if hasattr(old, replace):
7a6fd7cd16e9 Don't fail in the face of a lacking attribute when wrapping a Brett Cannon <brett@python.org> parents: 74643 diff changeset	146 setattr(new, replace, getattr(old, replace))
51433 43d3a95ed662 Create a simple substitute for functools.wraps to use in importlib._bootstrap. Brett Cannon <bcannon@gmail.com> parents: 51430 diff changeset	147 new.__dict__.update(old.__dict__)
43d3a95ed662 Create a simple substitute for functools.wraps to use in importlib._bootstrap. Brett Cannon <bcannon@gmail.com> parents: 51430 diff changeset	148
43d3a95ed662 Create a simple substitute for functools.wraps to use in importlib._bootstrap. Brett Cannon <bcannon@gmail.com> parents: 51430 diff changeset	149
76753 59b8a7fcb047 Properly mark names in importlib._bootstrap as private. Brett Cannon <brett@python.org> parents: 76586 diff changeset	150 _code_type = type(_wrap.__code__)
62756 34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	151
76331 d777f854a66e Issue #13959: Rename imp to _imp and add Lib/imp.py and begin Brett Cannon <brett@python.org> parents: 76330 diff changeset	152
76753 59b8a7fcb047 Properly mark names in importlib._bootstrap as private. Brett Cannon <brett@python.org> parents: 76586 diff changeset	153 def new_module(name):
76331 d777f854a66e Issue #13959: Rename imp to _imp and add Lib/imp.py and begin Brett Cannon <brett@python.org> parents: 76330 diff changeset	154 """Create a new module.
d777f854a66e Issue #13959: Rename imp to _imp and add Lib/imp.py and begin Brett Cannon <brett@python.org> parents: 76330 diff changeset	155
d777f854a66e Issue #13959: Rename imp to _imp and add Lib/imp.py and begin Brett Cannon <brett@python.org> parents: 76330 diff changeset	156 The module is not entered into sys.modules.
d777f854a66e Issue #13959: Rename imp to _imp and add Lib/imp.py and begin Brett Cannon <brett@python.org> parents: 76330 diff changeset	157
d777f854a66e Issue #13959: Rename imp to _imp and add Lib/imp.py and begin Brett Cannon <brett@python.org> parents: 76330 diff changeset	158 """
76755 c6a97506b2ee Jython-friendly tweak. Brett Cannon <brett@python.org> parents: 76754 diff changeset	159 return type(_io)(name)
76331 d777f854a66e Issue #13959: Rename imp to _imp and add Lib/imp.py and begin Brett Cannon <brett@python.org> parents: 76330 diff changeset	160
d777f854a66e Issue #13959: Rename imp to _imp and add Lib/imp.py and begin Brett Cannon <brett@python.org> parents: 76330 diff changeset	161
77013 edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	162 # Module-level locking ########################################################
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	163
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	164 # A dict mapping module names to weakrefs of _ModuleLock instances
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	165 _module_locks = {}
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	166 # A dict mapping thread ids to _ModuleLock instances
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	167 _blocking_on = {}
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	168
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	169
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	170 class _DeadlockError(RuntimeError):
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	171 pass
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	172
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	173
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	174 class _ModuleLock:
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	175 """A recursive lock implementation which is able to detect deadlocks
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	176 (e.g. thread 1 trying to take locks A then B, and thread 2 trying to
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	177 take locks B then A).
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	178 """
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	179
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	180 def __init__(self, name):
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	181 self.lock = _thread.allocate_lock()
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	182 self.wakeup = _thread.allocate_lock()
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	183 self.name = name
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	184 self.owner = None
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	185 self.count = 0
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	186 self.waiters = 0
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	187
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	188 def has_deadlock(self):
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	189 # Deadlock avoidance for concurrent circular imports.
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	190 me = _thread.get_ident()
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	191 tid = self.owner
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	192 while True:
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	193 lock = _blocking_on.get(tid)
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	194 if lock is None:
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	195 return False
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	196 tid = lock.owner
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	197 if tid == me:
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	198 return True
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	199
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	200 def acquire(self):
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	201 """
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	202 Acquire the module lock. If a potential deadlock is detected,
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	203 a _DeadlockError is raised.
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	204 Otherwise, the lock is always acquired and True is returned.
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	205 """
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	206 tid = _thread.get_ident()
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	207 _blocking_on[tid] = self
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	208 try:
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	209 while True:
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	210 with self.lock:
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	211 if self.count == 0 or self.owner == tid:
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	212 self.owner = tid
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	213 self.count += 1
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	214 return True
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	215 if self.has_deadlock():
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	216 raise _DeadlockError("deadlock detected by %r" % self)
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	217 if self.wakeup.acquire(False):
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	218 self.waiters += 1
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	219 # Wait for a release() call
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	220 self.wakeup.acquire()
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	221 self.wakeup.release()
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	222 finally:
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	223 del _blocking_on[tid]
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	224
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	225 def release(self):
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	226 tid = _thread.get_ident()
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	227 with self.lock:
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	228 if self.owner != tid:
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	229 raise RuntimeError("cannot release un-acquired lock")
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	230 assert self.count > 0
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	231 self.count -= 1
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	232 if self.count == 0:
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	233 self.owner = None
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	234 if self.waiters:
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	235 self.waiters -= 1
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	236 self.wakeup.release()
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	237
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	238 def __repr__(self):
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	239 return "_ModuleLock(%r) at %d" % (self.name, id(self))
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	240
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	241
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	242 class _DummyModuleLock:
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	243 """A simple _ModuleLock equivalent for Python builds without
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	244 multi-threading support."""
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	245
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	246 def __init__(self, name):
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	247 self.name = name
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	248 self.count = 0
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	249
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	250 def acquire(self):
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	251 self.count += 1
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	252 return True
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	253
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	254 def release(self):
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	255 if self.count == 0:
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	256 raise RuntimeError("cannot release un-acquired lock")
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	257 self.count -= 1
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	258
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	259 def __repr__(self):
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	260 return "_DummyModuleLock(%r) at %d" % (self.name, id(self))
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	261
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	262
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	263 # The following two functions are for consumption by Python/import.c.
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	264
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	265 def _get_module_lock(name):
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	266 """Get or create the module lock for a given module name.
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	267
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	268 Should only be called with the import lock taken."""
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	269 lock = None
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	270 if name in _module_locks:
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	271 lock = _module_locks[name]()
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	272 if lock is None:
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	273 if _thread is None:
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	274 lock = _DummyModuleLock(name)
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	275 else:
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	276 lock = _ModuleLock(name)
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	277 def cb(_):
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	278 del _module_locks[name]
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	279 _module_locks[name] = _weakref.ref(lock, cb)
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	280 return lock
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	281
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	282 def _lock_unlock_module(name):
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	283 """Release the global import lock, and acquires then release the
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	284 module lock for a given module name.
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	285 This is used to ensure a module is completely initialized, in the
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	286 event it is being imported by another thread.
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	287
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	288 Should only be called with the import lock taken."""
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	289 lock = _get_module_lock(name)
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	290 _imp.release_lock()
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	291 try:
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	292 lock.acquire()
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	293 except _DeadlockError:
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	294 # Concurrent circular import, we'll accept a partially initialized
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	295 # module object.
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	296 pass
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	297 else:
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	298 lock.release()
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	299
78344 62033490ca0f Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise Nick Coghlan <ncoghlan@gmail.com> parents: 78314 diff changeset	300 # Frame stripping magic ###############################################
77013 edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	301
78344 62033490ca0f Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise Nick Coghlan <ncoghlan@gmail.com> parents: 78314 diff changeset	302 def _call_with_frames_removed(f, args, *kwds):
62033490ca0f Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise Nick Coghlan <ncoghlan@gmail.com> parents: 78314 diff changeset	303 """remove_importlib_frames in import.c will always remove sequences
62033490ca0f Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise Nick Coghlan <ncoghlan@gmail.com> parents: 78314 diff changeset	304 of importlib frames that end with a call to this function
62033490ca0f Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise Nick Coghlan <ncoghlan@gmail.com> parents: 78314 diff changeset	305
62033490ca0f Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise Nick Coghlan <ncoghlan@gmail.com> parents: 78314 diff changeset	306 Use it instead of a normal call in places where including the importlib
62033490ca0f Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise Nick Coghlan <ncoghlan@gmail.com> parents: 78314 diff changeset	307 frames introduces unwanted noise into the traceback (e.g. when executing
62033490ca0f Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise Nick Coghlan <ncoghlan@gmail.com> parents: 78314 diff changeset	308 module code)
62033490ca0f Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise Nick Coghlan <ncoghlan@gmail.com> parents: 78314 diff changeset	309 """
62033490ca0f Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise Nick Coghlan <ncoghlan@gmail.com> parents: 78314 diff changeset	310 return f(args, *kwds)
62033490ca0f Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise Nick Coghlan <ncoghlan@gmail.com> parents: 78314 diff changeset	311
62033490ca0f Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise Nick Coghlan <ncoghlan@gmail.com> parents: 78314 diff changeset	312
62033490ca0f Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise Nick Coghlan <ncoghlan@gmail.com> parents: 78314 diff changeset	313 # Finder/loader utility code ###############################################
52140 1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap. Brett Cannon <bcannon@gmail.com> parents: 52139 diff changeset	314
78042 efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	315 """Magic word to reject .pyc files generated by other Python versions.
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	316 It should change for each incompatible change to the bytecode.
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	317
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	318 The value of CR and LF is incorporated so if you ever read or write
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	319 a .pyc file in text mode the magic number will be wrong; also, the
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	320 Apple MPW compiler swaps their values, botching string constants.
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	321
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	322 The magic numbers must be spaced apart at least 2 values, as the
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	323 -U interpeter flag will cause MAGIC+1 being used. They have been
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	324 odd numbers for some time now.
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	325
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	326 There were a variety of old schemes for setting the magic number.
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	327 The current working scheme is to increment the previous value by
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	328 10.
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	329
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	330 Starting with the adoption of PEP 3147 in Python 3.2, every bump in magic
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	331 number also includes a new "magic tag", i.e. a human readable string used
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	332 to represent the magic number in __pycache__ directories. When you change
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	333 the magic number, you must also set a new unique magic tag. Generally this
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	334 can be named after the Python major version of the magic number bump, but
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	335 it can really be anything, as long as it's different than anything else
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	336 that's come before. The tags are included in the following table, starting
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	337 with Python 3.2a0.
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	338
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	339 Known values:
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	340 Python 1.5: 20121
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	341 Python 1.5.1: 20121
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	342 Python 1.5.2: 20121
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	343 Python 1.6: 50428
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	344 Python 2.0: 50823
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	345 Python 2.0.1: 50823
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	346 Python 2.1: 60202
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	347 Python 2.1.1: 60202
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	348 Python 2.1.2: 60202
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	349 Python 2.2: 60717
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	350 Python 2.3a0: 62011
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	351 Python 2.3a0: 62021
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	352 Python 2.3a0: 62011 (!)
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	353 Python 2.4a0: 62041
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	354 Python 2.4a3: 62051
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	355 Python 2.4b1: 62061
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	356 Python 2.5a0: 62071
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	357 Python 2.5a0: 62081 (ast-branch)
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	358 Python 2.5a0: 62091 (with)
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	359 Python 2.5a0: 62092 (changed WITH_CLEANUP opcode)
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	360 Python 2.5b3: 62101 (fix wrong code: for x, in ...)
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	361 Python 2.5b3: 62111 (fix wrong code: x += yield)
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	362 Python 2.5c1: 62121 (fix wrong lnotab with for loops and
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	363 storing constants that should have been removed)
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	364 Python 2.5c2: 62131 (fix wrong code: for x, in ... in listcomp/genexp)
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	365 Python 2.6a0: 62151 (peephole optimizations and STORE_MAP opcode)
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	366 Python 2.6a1: 62161 (WITH_CLEANUP optimization)
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	367 Python 3000: 3000
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	368 3010 (removed UNARY_CONVERT)
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	369 3020 (added BUILD_SET)
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	370 3030 (added keyword-only parameters)
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	371 3040 (added signature annotations)
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	372 3050 (print becomes a function)
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	373 3060 (PEP 3115 metaclass syntax)
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	374 3061 (string literals become unicode)
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	375 3071 (PEP 3109 raise changes)
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	376 3081 (PEP 3137 make __file__ and __name__ unicode)
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	377 3091 (kill str8 interning)
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	378 3101 (merge from 2.6a0, see 62151)
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	379 3103 (__file__ points to source file)
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	380 Python 3.0a4: 3111 (WITH_CLEANUP optimization).
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	381 Python 3.0a5: 3131 (lexical exception stacking, including POP_EXCEPT)
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	382 Python 3.1a0: 3141 (optimize list, set and dict comprehensions:
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	383 change LIST_APPEND and SET_ADD, add MAP_ADD)
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	384 Python 3.1a0: 3151 (optimize conditional branches:
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	385 introduce POP_JUMP_IF_FALSE and POP_JUMP_IF_TRUE)
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	386 Python 3.2a0: 3160 (add SETUP_WITH)
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	387 tag: cpython-32
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	388 Python 3.2a1: 3170 (add DUP_TOP_TWO, remove DUP_TOPX and ROT_FOUR)
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	389 tag: cpython-32
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	390 Python 3.2a2 3180 (add DELETE_DEREF)
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	391 Python 3.3a0 3190 __class__ super closure changed
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	392 Python 3.3a0 3200 (__qualname__ added)
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	393 3210 (added size modulo 2**32 to the pyc header)
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	394 Python 3.3a1 3220 (changed PEP 380 implementation)
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	395 Python 3.3a4 3230 (revert changes to implicit __class__ closure)
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	396
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	397 MAGIC must change whenever the bytecode emitted by the compiler may no
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	398 longer be understood by older implementations of the eval loop (usually
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	399 due to the addition of new opcodes).
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	400
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	401 """
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	402 _RAW_MAGIC_NUMBER = 3230 \| ord('\r') << 16 \| ord('\n') << 24
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	403 _MAGIC_BYTES = bytes(_RAW_MAGIC_NUMBER >> n & 0xff for n in range(0, 25, 8))
efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	404
76753 59b8a7fcb047 Properly mark names in importlib._bootstrap as private. Brett Cannon <brett@python.org> parents: 76586 diff changeset	405 _PYCACHE = '__pycache__'
76450 b773a751c2e7 Issue #13959: Re-implement imp.cache_from_source() in Lib/imp.py. Brett Cannon <brett@python.org> parents: 76449 diff changeset	406
76874 b81ddaf0db47 Issue #13959: Deprecate imp.get_suffixes() for new attributes on Brett Cannon <brett@python.org> parents: 76826 diff changeset	407 SOURCE_SUFFIXES = ['.py'] # _setup() adds .pyw as needed.
76451 ea46ebba8a0f Issue #13959: Re-implement imp.source_from_cache() in Lib/imp.py. Brett Cannon <brett@python.org> parents: 76450 diff changeset	408
76874 b81ddaf0db47 Issue #13959: Deprecate imp.get_suffixes() for new attributes on Brett Cannon <brett@python.org> parents: 76826 diff changeset	409 DEBUG_BYTECODE_SUFFIXES = ['.pyc']
b81ddaf0db47 Issue #13959: Deprecate imp.get_suffixes() for new attributes on Brett Cannon <brett@python.org> parents: 76826 diff changeset	410 OPTIMIZED_BYTECODE_SUFFIXES = ['.pyo']
b81ddaf0db47 Issue #13959: Deprecate imp.get_suffixes() for new attributes on Brett Cannon <brett@python.org> parents: 76826 diff changeset	411 if __debug__:
b81ddaf0db47 Issue #13959: Deprecate imp.get_suffixes() for new attributes on Brett Cannon <brett@python.org> parents: 76826 diff changeset	412 BYTECODE_SUFFIXES = DEBUG_BYTECODE_SUFFIXES
b81ddaf0db47 Issue #13959: Deprecate imp.get_suffixes() for new attributes on Brett Cannon <brett@python.org> parents: 76826 diff changeset	413 else:
b81ddaf0db47 Issue #13959: Deprecate imp.get_suffixes() for new attributes on Brett Cannon <brett@python.org> parents: 76826 diff changeset	414 BYTECODE_SUFFIXES = OPTIMIZED_BYTECODE_SUFFIXES
76450 b773a751c2e7 Issue #13959: Re-implement imp.cache_from_source() in Lib/imp.py. Brett Cannon <brett@python.org> parents: 76449 diff changeset	415
76753 59b8a7fcb047 Properly mark names in importlib._bootstrap as private. Brett Cannon <brett@python.org> parents: 76586 diff changeset	416 def cache_from_source(path, debug_override=None):
76450 b773a751c2e7 Issue #13959: Re-implement imp.cache_from_source() in Lib/imp.py. Brett Cannon <brett@python.org> parents: 76449 diff changeset	417 """Given the path to a .py file, return the path to its .pyc/.pyo file.
b773a751c2e7 Issue #13959: Re-implement imp.cache_from_source() in Lib/imp.py. Brett Cannon <brett@python.org> parents: 76449 diff changeset	418
b773a751c2e7 Issue #13959: Re-implement imp.cache_from_source() in Lib/imp.py. Brett Cannon <brett@python.org> parents: 76449 diff changeset	419 The .py file does not need to exist; this simply returns the path to the
b773a751c2e7 Issue #13959: Re-implement imp.cache_from_source() in Lib/imp.py. Brett Cannon <brett@python.org> parents: 76449 diff changeset	420 .pyc/.pyo file calculated as if the .py file were imported. The extension
b773a751c2e7 Issue #13959: Re-implement imp.cache_from_source() in Lib/imp.py. Brett Cannon <brett@python.org> parents: 76449 diff changeset	421 will be .pyc unless __debug__ is not defined, then it will be .pyo.
b773a751c2e7 Issue #13959: Re-implement imp.cache_from_source() in Lib/imp.py. Brett Cannon <brett@python.org> parents: 76449 diff changeset	422
b773a751c2e7 Issue #13959: Re-implement imp.cache_from_source() in Lib/imp.py. Brett Cannon <brett@python.org> parents: 76449 diff changeset	423 If debug_override is not None, then it must be a boolean and is taken as
b773a751c2e7 Issue #13959: Re-implement imp.cache_from_source() in Lib/imp.py. Brett Cannon <brett@python.org> parents: 76449 diff changeset	424 the value of __debug__ instead.
b773a751c2e7 Issue #13959: Re-implement imp.cache_from_source() in Lib/imp.py. Brett Cannon <brett@python.org> parents: 76449 diff changeset	425
78026 e86330669bb5 Issue #15056: imp.cache_from_source() and source_from_cache() raise Brett Cannon <brett@python.org> parents: 78010 diff changeset	426 If sys.implementation.cache_tag is None then NotImplementedError is raised.
e86330669bb5 Issue #15056: imp.cache_from_source() and source_from_cache() raise Brett Cannon <brett@python.org> parents: 78010 diff changeset	427
76450 b773a751c2e7 Issue #13959: Re-implement imp.cache_from_source() in Lib/imp.py. Brett Cannon <brett@python.org> parents: 76449 diff changeset	428 """
b773a751c2e7 Issue #13959: Re-implement imp.cache_from_source() in Lib/imp.py. Brett Cannon <brett@python.org> parents: 76449 diff changeset	429 debug = __debug__ if debug_override is None else debug_override
76874 b81ddaf0db47 Issue #13959: Deprecate imp.get_suffixes() for new attributes on Brett Cannon <brett@python.org> parents: 76826 diff changeset	430 if debug:
b81ddaf0db47 Issue #13959: Deprecate imp.get_suffixes() for new attributes on Brett Cannon <brett@python.org> parents: 76826 diff changeset	431 suffixes = DEBUG_BYTECODE_SUFFIXES
b81ddaf0db47 Issue #13959: Deprecate imp.get_suffixes() for new attributes on Brett Cannon <brett@python.org> parents: 76826 diff changeset	432 else:
b81ddaf0db47 Issue #13959: Deprecate imp.get_suffixes() for new attributes on Brett Cannon <brett@python.org> parents: 76826 diff changeset	433 suffixes = OPTIMIZED_BYTECODE_SUFFIXES
76450 b773a751c2e7 Issue #13959: Re-implement imp.cache_from_source() in Lib/imp.py. Brett Cannon <brett@python.org> parents: 76449 diff changeset	434 head, tail = _path_split(path)
b773a751c2e7 Issue #13959: Re-implement imp.cache_from_source() in Lib/imp.py. Brett Cannon <brett@python.org> parents: 76449 diff changeset	435 base_filename, sep, _ = tail.partition('.')
78026 e86330669bb5 Issue #15056: imp.cache_from_source() and source_from_cache() raise Brett Cannon <brett@python.org> parents: 78010 diff changeset	436 tag = sys.implementation.cache_tag
e86330669bb5 Issue #15056: imp.cache_from_source() and source_from_cache() raise Brett Cannon <brett@python.org> parents: 78010 diff changeset	437 if tag is None:
e86330669bb5 Issue #15056: imp.cache_from_source() and source_from_cache() raise Brett Cannon <brett@python.org> parents: 78010 diff changeset	438 raise NotImplementedError('sys.implementation.cache_tag is None')
e86330669bb5 Issue #15056: imp.cache_from_source() and source_from_cache() raise Brett Cannon <brett@python.org> parents: 78010 diff changeset	439 filename = ''.join([base_filename, sep, tag, suffixes[0]])
76753 59b8a7fcb047 Properly mark names in importlib._bootstrap as private. Brett Cannon <brett@python.org> parents: 76586 diff changeset	440 return _path_join(head, _PYCACHE, filename)
76450 b773a751c2e7 Issue #13959: Re-implement imp.cache_from_source() in Lib/imp.py. Brett Cannon <brett@python.org> parents: 76449 diff changeset	441
b773a751c2e7 Issue #13959: Re-implement imp.cache_from_source() in Lib/imp.py. Brett Cannon <brett@python.org> parents: 76449 diff changeset	442
78078 9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	443 def source_from_cache(path):
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	444 """Given the path to a .pyc./.pyo file, return the path to its .py file.
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	445
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	446 The .pyc/.pyo file does not need to exist; this simply returns the path to
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	447 the .py file calculated to correspond to the .pyc/.pyo file. If path does
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	448 not conform to PEP 3147 format, ValueError will be raised. If
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	449 sys.implementation.cache_tag is None then NotImplementedError is raised.
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	450
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	451 """
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	452 if sys.implementation.cache_tag is None:
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	453 raise NotImplementedError('sys.implementation.cache_tag is None')
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	454 head, pycache_filename = _path_split(path)
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	455 head, pycache = _path_split(head)
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	456 if pycache != _PYCACHE:
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	457 raise ValueError('{} not bottom-level directory in '
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	458 '{!r}'.format(_PYCACHE, path))
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	459 if pycache_filename.count('.') != 2:
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	460 raise ValueError('expected only 2 dots in '
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	461 '{!r}'.format(pycache_filename))
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	462 base_filename = pycache_filename.partition('.')[0]
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	463 return _path_join(head, base_filename + SOURCE_SUFFIXES[0])
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	464
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	465
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	466 def _get_sourcefile(bytecode_path):
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	467 """Convert a bytecode file path to a source path (if possible).
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	468
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	469 This function exists purely for backwards-compatibility for
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	470 PyImport_ExecCodeModuleWithFilenames() in the C API.
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	471
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	472 """
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	473 if len(bytecode_path) == 0:
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	474 return None
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	475 rest, _, extension = bytecode_path.rparition('.')
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	476 if not rest or extension.lower()[-3:-1] != '.py':
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	477 return bytecode_path
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	478
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	479 try:
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	480 source_path = source_from_cache(bytecode_path)
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	481 except (NotImplementedError, ValueError):
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	482 source_path = bytcode_path[-1:]
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	483
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	484 return source_path if _path_isfile(source_stats) else bytecode_path
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	485
9e164b404983 Issues #15169, #14599: Make PyImport_ExecCodeModuleWithPathnames() use Brett Cannon <brett@python.org> parents: 78051 diff changeset	486
76753 59b8a7fcb047 Properly mark names in importlib._bootstrap as private. Brett Cannon <brett@python.org> parents: 76586 diff changeset	487 def _verbose_message(message, *args):
76303 2dd046be2c88 Issue #2377: Make importlib the implementation of __import__(). Brett Cannon <brett@python.org> parents: 76280 diff changeset	488 """Print the message to stderr if -v/PYTHONVERBOSE is turned on."""
2dd046be2c88 Issue #2377: Make importlib the implementation of __import__(). Brett Cannon <brett@python.org> parents: 76280 diff changeset	489 if sys.flags.verbose:
76327 1ca10eb58483 utilize startswith(tuple) Philip Jenvey <pjenvey@underboss.org> parents: 76312 diff changeset	490 if not message.startswith(('#', 'import ')):
76303 2dd046be2c88 Issue #2377: Make importlib the implementation of __import__(). Brett Cannon <brett@python.org> parents: 76280 diff changeset	491 message = '# ' + message
2dd046be2c88 Issue #2377: Make importlib the implementation of __import__(). Brett Cannon <brett@python.org> parents: 76280 diff changeset	492 print(message.format(*args), file=sys.stderr)
2dd046be2c88 Issue #2377: Make importlib the implementation of __import__(). Brett Cannon <brett@python.org> parents: 76280 diff changeset	493
75048 bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function. Antoine Pitrou <solipsis@pitrou.net> parents: 75045 diff changeset	494
52002 76a579b6bade Rename importlib.util.set___package__ to set_package. Brett Cannon <bcannon@gmail.com> parents: 51995 diff changeset	495 def set_package(fxn):
51430 5cdc5f0c134c Move importlib completely over to using rpartition and accepting the empty Brett Cannon <bcannon@gmail.com> parents: 51429 diff changeset	496 """Set __package__ on the returned module."""
74449 9935f7837c4b Issue #13588: Rename decorators in importlib. Brett Cannon <brett@python.org> parents: 74358 diff changeset	497 def set_package_wrapper(args, *kwargs):
51430 5cdc5f0c134c Move importlib completely over to using rpartition and accepting the empty Brett Cannon <bcannon@gmail.com> parents: 51429 diff changeset	498 module = fxn(args, *kwargs)
77964 37850f083af5 Minor refactoring in importlib._bootstrap, and fix the '_wrap' docstring. Florent Xicluna <florent.xicluna@gmail.com> parents: 77917 diff changeset	499 if getattr(module, '__package__', None) is None:
51430 5cdc5f0c134c Move importlib completely over to using rpartition and accepting the empty Brett Cannon <bcannon@gmail.com> parents: 51429 diff changeset	500 module.__package__ = module.__name__
5cdc5f0c134c Move importlib completely over to using rpartition and accepting the empty Brett Cannon <bcannon@gmail.com> parents: 51429 diff changeset	501 if not hasattr(module, '__path__'):
5cdc5f0c134c Move importlib completely over to using rpartition and accepting the empty Brett Cannon <bcannon@gmail.com> parents: 51429 diff changeset	502 module.__package__ = module.__package__.rpartition('.')[0]
5cdc5f0c134c Move importlib completely over to using rpartition and accepting the empty Brett Cannon <bcannon@gmail.com> parents: 51429 diff changeset	503 return module
74449 9935f7837c4b Issue #13588: Rename decorators in importlib. Brett Cannon <brett@python.org> parents: 74358 diff changeset	504 _wrap(set_package_wrapper, fxn)
9935f7837c4b Issue #13588: Rename decorators in importlib. Brett Cannon <brett@python.org> parents: 74358 diff changeset	505 return set_package_wrapper
51430 5cdc5f0c134c Move importlib completely over to using rpartition and accepting the empty Brett Cannon <bcannon@gmail.com> parents: 51429 diff changeset	506
5cdc5f0c134c Move importlib completely over to using rpartition and accepting the empty Brett Cannon <bcannon@gmail.com> parents: 51429 diff changeset	507
52115 afd7118c8ccf Implement importlib.util.set_loader: a decorator to automatically set Brett Cannon <bcannon@gmail.com> parents: 52112 diff changeset	508 def set_loader(fxn):
afd7118c8ccf Implement importlib.util.set_loader: a decorator to automatically set Brett Cannon <bcannon@gmail.com> parents: 52112 diff changeset	509 """Set __loader__ on the returned module."""
74449 9935f7837c4b Issue #13588: Rename decorators in importlib. Brett Cannon <brett@python.org> parents: 74358 diff changeset	510 def set_loader_wrapper(self, args, *kwargs):
52115 afd7118c8ccf Implement importlib.util.set_loader: a decorator to automatically set Brett Cannon <bcannon@gmail.com> parents: 52112 diff changeset	511 module = fxn(self, args, *kwargs)
afd7118c8ccf Implement importlib.util.set_loader: a decorator to automatically set Brett Cannon <bcannon@gmail.com> parents: 52112 diff changeset	512 if not hasattr(module, '__loader__'):
afd7118c8ccf Implement importlib.util.set_loader: a decorator to automatically set Brett Cannon <bcannon@gmail.com> parents: 52112 diff changeset	513 module.__loader__ = self
afd7118c8ccf Implement importlib.util.set_loader: a decorator to automatically set Brett Cannon <bcannon@gmail.com> parents: 52112 diff changeset	514 return module
74449 9935f7837c4b Issue #13588: Rename decorators in importlib. Brett Cannon <brett@python.org> parents: 74358 diff changeset	515 _wrap(set_loader_wrapper, fxn)
9935f7837c4b Issue #13588: Rename decorators in importlib. Brett Cannon <brett@python.org> parents: 74358 diff changeset	516 return set_loader_wrapper
52115 afd7118c8ccf Implement importlib.util.set_loader: a decorator to automatically set Brett Cannon <bcannon@gmail.com> parents: 52112 diff changeset	517
afd7118c8ccf Implement importlib.util.set_loader: a decorator to automatically set Brett Cannon <bcannon@gmail.com> parents: 52112 diff changeset	518
52140 1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap. Brett Cannon <bcannon@gmail.com> parents: 52139 diff changeset	519 def module_for_loader(fxn):
1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap. Brett Cannon <bcannon@gmail.com> parents: 52139 diff changeset	520 """Decorator to handle selecting the proper module for loaders.
1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap. Brett Cannon <bcannon@gmail.com> parents: 52139 diff changeset	521
52164 0b19ba318071 Clean up docstring from importlib.util.module_for_loader. Brett Cannon <bcannon@gmail.com> parents: 52142 diff changeset	522 The decorated function is passed the module to use instead of the module
0b19ba318071 Clean up docstring from importlib.util.module_for_loader. Brett Cannon <bcannon@gmail.com> parents: 52142 diff changeset	523 name. The module passed in to the function is either from sys.modules if
76586 496c68f90a03 Issue #14646: __import__() now sets __loader__ if need be. Brett Cannon <brett@python.org> parents: 76582 diff changeset	524 it already exists or is a new module. If the module is new, then __name__
496c68f90a03 Issue #14646: __import__() now sets __loader__ if need be. Brett Cannon <brett@python.org> parents: 76582 diff changeset	525 is set the first argument to the method, __loader__ is set to self, and
496c68f90a03 Issue #14646: __import__() now sets __loader__ if need be. Brett Cannon <brett@python.org> parents: 76582 diff changeset	526 __package__ is set accordingly (if self.is_package() is defined) will be set
496c68f90a03 Issue #14646: __import__() now sets __loader__ if need be. Brett Cannon <brett@python.org> parents: 76582 diff changeset	527 before it is passed to the decorated function (if self.is_package() does
496c68f90a03 Issue #14646: __import__() now sets __loader__ if need be. Brett Cannon <brett@python.org> parents: 76582 diff changeset	528 not work for the module it will be set post-load).
496c68f90a03 Issue #14646: __import__() now sets __loader__ if need be. Brett Cannon <brett@python.org> parents: 76582 diff changeset	529
496c68f90a03 Issue #14646: __import__() now sets __loader__ if need be. Brett Cannon <brett@python.org> parents: 76582 diff changeset	530 If an exception is raised and the decorator created the module it is
496c68f90a03 Issue #14646: __import__() now sets __loader__ if need be. Brett Cannon <brett@python.org> parents: 76582 diff changeset	531 subsequently removed from sys.modules.
52140 1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap. Brett Cannon <bcannon@gmail.com> parents: 52139 diff changeset	532
52164 0b19ba318071 Clean up docstring from importlib.util.module_for_loader. Brett Cannon <bcannon@gmail.com> parents: 52142 diff changeset	533 The decorator assumes that the decorated function takes the module name as
0b19ba318071 Clean up docstring from importlib.util.module_for_loader. Brett Cannon <bcannon@gmail.com> parents: 52142 diff changeset	534 the second argument.
52140 1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap. Brett Cannon <bcannon@gmail.com> parents: 52139 diff changeset	535
1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap. Brett Cannon <bcannon@gmail.com> parents: 52139 diff changeset	536 """
74449 9935f7837c4b Issue #13588: Rename decorators in importlib. Brett Cannon <brett@python.org> parents: 74358 diff changeset	537 def module_for_loader_wrapper(self, fullname, args, *kwargs):
52140 1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap. Brett Cannon <bcannon@gmail.com> parents: 52139 diff changeset	538 module = sys.modules.get(fullname)
76384 c1399cf7bd6a Issue #12599: Be more strict in accepting None vs. a false-like object Brett Cannon <brett@python.org> parents: 76331 diff changeset	539 is_reload = module is not None
52140 1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap. Brett Cannon <bcannon@gmail.com> parents: 52139 diff changeset	540 if not is_reload:
1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap. Brett Cannon <bcannon@gmail.com> parents: 52139 diff changeset	541 # This must be done before open() is called as the 'io' module
1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap. Brett Cannon <bcannon@gmail.com> parents: 52139 diff changeset	542 # implicitly imports 'locale' and would otherwise trigger an
1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap. Brett Cannon <bcannon@gmail.com> parents: 52139 diff changeset	543 # infinite loop.
76753 59b8a7fcb047 Properly mark names in importlib._bootstrap as private. Brett Cannon <brett@python.org> parents: 76586 diff changeset	544 module = new_module(fullname)
52140 1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap. Brett Cannon <bcannon@gmail.com> parents: 52139 diff changeset	545 sys.modules[fullname] = module
76586 496c68f90a03 Issue #14646: __import__() now sets __loader__ if need be. Brett Cannon <brett@python.org> parents: 76582 diff changeset	546 module.__loader__ = self
496c68f90a03 Issue #14646: __import__() now sets __loader__ if need be. Brett Cannon <brett@python.org> parents: 76582 diff changeset	547 try:
496c68f90a03 Issue #14646: __import__() now sets __loader__ if need be. Brett Cannon <brett@python.org> parents: 76582 diff changeset	548 is_package = self.is_package(fullname)
496c68f90a03 Issue #14646: __import__() now sets __loader__ if need be. Brett Cannon <brett@python.org> parents: 76582 diff changeset	549 except (ImportError, AttributeError):
496c68f90a03 Issue #14646: __import__() now sets __loader__ if need be. Brett Cannon <brett@python.org> parents: 76582 diff changeset	550 pass
496c68f90a03 Issue #14646: __import__() now sets __loader__ if need be. Brett Cannon <brett@python.org> parents: 76582 diff changeset	551 else:
496c68f90a03 Issue #14646: __import__() now sets __loader__ if need be. Brett Cannon <brett@python.org> parents: 76582 diff changeset	552 if is_package:
496c68f90a03 Issue #14646: __import__() now sets __loader__ if need be. Brett Cannon <brett@python.org> parents: 76582 diff changeset	553 module.__package__ = fullname
496c68f90a03 Issue #14646: __import__() now sets __loader__ if need be. Brett Cannon <brett@python.org> parents: 76582 diff changeset	554 else:
496c68f90a03 Issue #14646: __import__() now sets __loader__ if need be. Brett Cannon <brett@python.org> parents: 76582 diff changeset	555 module.__package__ = fullname.rpartition('.')[0]
52140 1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap. Brett Cannon <bcannon@gmail.com> parents: 52139 diff changeset	556 try:
77013 edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	557 module.__initializing__ = True
76586 496c68f90a03 Issue #14646: __import__() now sets __loader__ if need be. Brett Cannon <brett@python.org> parents: 76582 diff changeset	558 # If __package__ was not set above, __import__() will do it later.
62756 34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	559 return fxn(self, module, args, *kwargs)
52140 1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap. Brett Cannon <bcannon@gmail.com> parents: 52139 diff changeset	560 except:
1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap. Brett Cannon <bcannon@gmail.com> parents: 52139 diff changeset	561 if not is_reload:
1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap. Brett Cannon <bcannon@gmail.com> parents: 52139 diff changeset	562 del sys.modules[fullname]
1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap. Brett Cannon <bcannon@gmail.com> parents: 52139 diff changeset	563 raise
77013 edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	564 finally:
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	565 module.__initializing__ = False
74449 9935f7837c4b Issue #13588: Rename decorators in importlib. Brett Cannon <brett@python.org> parents: 74358 diff changeset	566 _wrap(module_for_loader_wrapper, fxn)
9935f7837c4b Issue #13588: Rename decorators in importlib. Brett Cannon <brett@python.org> parents: 74358 diff changeset	567 return module_for_loader_wrapper
52140 1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap. Brett Cannon <bcannon@gmail.com> parents: 52139 diff changeset	568
1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap. Brett Cannon <bcannon@gmail.com> parents: 52139 diff changeset	569
1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap. Brett Cannon <bcannon@gmail.com> parents: 52139 diff changeset	570 def _check_name(method):
1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap. Brett Cannon <bcannon@gmail.com> parents: 52139 diff changeset	571 """Decorator to verify that the module being requested matches the one the
1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap. Brett Cannon <bcannon@gmail.com> parents: 52139 diff changeset	572 loader can handle.
1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap. Brett Cannon <bcannon@gmail.com> parents: 52139 diff changeset	573
1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap. Brett Cannon <bcannon@gmail.com> parents: 52139 diff changeset	574 The first argument (self) must define _name which the second argument is
68488 0991b40e895d #11515: fix several typos. Patch by Piotr Kasprzyk. Ezio Melotti <ezio.melotti@gmail.com> parents: 58960 diff changeset	575 compared against. If the comparison fails then ImportError is raised.
52140 1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap. Brett Cannon <bcannon@gmail.com> parents: 52139 diff changeset	576
1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap. Brett Cannon <bcannon@gmail.com> parents: 52139 diff changeset	577 """
76877 626d5c6fbd95 Issue #13959: Have Brett Cannon <brett@python.org> parents: 76874 diff changeset	578 def _check_name_wrapper(self, name=None, args, *kwargs):
626d5c6fbd95 Issue #13959: Have Brett Cannon <brett@python.org> parents: 76874 diff changeset	579 if name is None:
626d5c6fbd95 Issue #13959: Have Brett Cannon <brett@python.org> parents: 76874 diff changeset	580 name = self.name
626d5c6fbd95 Issue #13959: Have Brett Cannon <brett@python.org> parents: 76874 diff changeset	581 elif self.name != name:
76280 c071af95772a Have importlib take advantage of ImportError's new 'name' and 'path' Brett Cannon <brett@python.org> parents: 76081 diff changeset	582 raise ImportError("loader cannot handle %s" % name, name=name)
52140 1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap. Brett Cannon <bcannon@gmail.com> parents: 52139 diff changeset	583 return method(self, name, args, *kwargs)
74449 9935f7837c4b Issue #13588: Rename decorators in importlib. Brett Cannon <brett@python.org> parents: 74358 diff changeset	584 _wrap(_check_name_wrapper, method)
9935f7837c4b Issue #13588: Rename decorators in importlib. Brett Cannon <brett@python.org> parents: 74358 diff changeset	585 return _check_name_wrapper
52140 1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap. Brett Cannon <bcannon@gmail.com> parents: 52139 diff changeset	586
1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap. Brett Cannon <bcannon@gmail.com> parents: 52139 diff changeset	587
52166 69cd615b76fb Implement InspectLoader for BuiltinImporter. Brett Cannon <bcannon@gmail.com> parents: 52165 diff changeset	588 def _requires_builtin(fxn):
69cd615b76fb Implement InspectLoader for BuiltinImporter. Brett Cannon <bcannon@gmail.com> parents: 52165 diff changeset	589 """Decorator to verify the named module is built-in."""
74449 9935f7837c4b Issue #13588: Rename decorators in importlib. Brett Cannon <brett@python.org> parents: 74358 diff changeset	590 def _requires_builtin_wrapper(self, fullname):
52166 69cd615b76fb Implement InspectLoader for BuiltinImporter. Brett Cannon <bcannon@gmail.com> parents: 52165 diff changeset	591 if fullname not in sys.builtin_module_names:
77964 37850f083af5 Minor refactoring in importlib._bootstrap, and fix the '_wrap' docstring. Florent Xicluna <florent.xicluna@gmail.com> parents: 77917 diff changeset	592 raise ImportError("{} is not a built-in module".format(fullname),
76280 c071af95772a Have importlib take advantage of ImportError's new 'name' and 'path' Brett Cannon <brett@python.org> parents: 76081 diff changeset	593 name=fullname)
52166 69cd615b76fb Implement InspectLoader for BuiltinImporter. Brett Cannon <bcannon@gmail.com> parents: 52165 diff changeset	594 return fxn(self, fullname)
74449 9935f7837c4b Issue #13588: Rename decorators in importlib. Brett Cannon <brett@python.org> parents: 74358 diff changeset	595 _wrap(_requires_builtin_wrapper, fxn)
9935f7837c4b Issue #13588: Rename decorators in importlib. Brett Cannon <brett@python.org> parents: 74358 diff changeset	596 return _requires_builtin_wrapper
52166 69cd615b76fb Implement InspectLoader for BuiltinImporter. Brett Cannon <bcannon@gmail.com> parents: 52165 diff changeset	597
69cd615b76fb Implement InspectLoader for BuiltinImporter. Brett Cannon <bcannon@gmail.com> parents: 52165 diff changeset	598
52168 70f5f9113ae1 Implement InspectLoader for FrozenImporter. Brett Cannon <bcannon@gmail.com> parents: 52166 diff changeset	599 def _requires_frozen(fxn):
70f5f9113ae1 Implement InspectLoader for FrozenImporter. Brett Cannon <bcannon@gmail.com> parents: 52166 diff changeset	600 """Decorator to verify the named module is frozen."""
74449 9935f7837c4b Issue #13588: Rename decorators in importlib. Brett Cannon <brett@python.org> parents: 74358 diff changeset	601 def _requires_frozen_wrapper(self, fullname):
76331 d777f854a66e Issue #13959: Rename imp to _imp and add Lib/imp.py and begin Brett Cannon <brett@python.org> parents: 76330 diff changeset	602 if not _imp.is_frozen(fullname):
77964 37850f083af5 Minor refactoring in importlib._bootstrap, and fix the '_wrap' docstring. Florent Xicluna <florent.xicluna@gmail.com> parents: 77917 diff changeset	603 raise ImportError("{} is not a frozen module".format(fullname),
76280 c071af95772a Have importlib take advantage of ImportError's new 'name' and 'path' Brett Cannon <brett@python.org> parents: 76081 diff changeset	604 name=fullname)
52168 70f5f9113ae1 Implement InspectLoader for FrozenImporter. Brett Cannon <bcannon@gmail.com> parents: 52166 diff changeset	605 return fxn(self, fullname)
74449 9935f7837c4b Issue #13588: Rename decorators in importlib. Brett Cannon <brett@python.org> parents: 74358 diff changeset	606 _wrap(_requires_frozen_wrapper, fxn)
9935f7837c4b Issue #13588: Rename decorators in importlib. Brett Cannon <brett@python.org> parents: 74358 diff changeset	607 return _requires_frozen_wrapper
52168 70f5f9113ae1 Implement InspectLoader for FrozenImporter. Brett Cannon <bcannon@gmail.com> parents: 52166 diff changeset	608
70f5f9113ae1 Implement InspectLoader for FrozenImporter. Brett Cannon <bcannon@gmail.com> parents: 52166 diff changeset	609
78489 e7a67f1bf604 Issue #15502: Refactor some code. Brett Cannon <brett@python.org> parents: 78487 diff changeset	610 def _find_module_shim(self, fullname):
e7a67f1bf604 Issue #15502: Refactor some code. Brett Cannon <brett@python.org> parents: 78487 diff changeset	611 """Try to find a loader for the specified module by delegating to
e7a67f1bf604 Issue #15502: Refactor some code. Brett Cannon <brett@python.org> parents: 78487 diff changeset	612 self.find_loader()."""
e7a67f1bf604 Issue #15502: Refactor some code. Brett Cannon <brett@python.org> parents: 78487 diff changeset	613 # Call find_loader(). If it returns a string (indicating this
e7a67f1bf604 Issue #15502: Refactor some code. Brett Cannon <brett@python.org> parents: 78487 diff changeset	614 # is a namespace package portion), generate a warning and
e7a67f1bf604 Issue #15502: Refactor some code. Brett Cannon <brett@python.org> parents: 78487 diff changeset	615 # return None.
e7a67f1bf604 Issue #15502: Refactor some code. Brett Cannon <brett@python.org> parents: 78487 diff changeset	616 loader, portions = self.find_loader(fullname)
e7a67f1bf604 Issue #15502: Refactor some code. Brett Cannon <brett@python.org> parents: 78487 diff changeset	617 if loader is None and len(portions):
e7a67f1bf604 Issue #15502: Refactor some code. Brett Cannon <brett@python.org> parents: 78487 diff changeset	618 msg = "Not importing directory {}: missing __init__"
e7a67f1bf604 Issue #15502: Refactor some code. Brett Cannon <brett@python.org> parents: 78487 diff changeset	619 _warnings.warn(msg.format(portions[0]), ImportWarning)
e7a67f1bf604 Issue #15502: Refactor some code. Brett Cannon <brett@python.org> parents: 78487 diff changeset	620 return loader
e7a67f1bf604 Issue #15502: Refactor some code. Brett Cannon <brett@python.org> parents: 78487 diff changeset	621
e7a67f1bf604 Issue #15502: Refactor some code. Brett Cannon <brett@python.org> parents: 78487 diff changeset	622
e7a67f1bf604 Issue #15502: Refactor some code. Brett Cannon <brett@python.org> parents: 78487 diff changeset	623
e7a67f1bf604 Issue #15502: Refactor some code. Brett Cannon <brett@python.org> parents: 78487 diff changeset	624
52141 f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	625 # Loaders #####################################################################
52140 1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap. Brett Cannon <bcannon@gmail.com> parents: 52139 diff changeset	626
50983 0940a4f2506e Add importlib.machinery with its first tenants, BuitinImporter and Brett Cannon <bcannon@gmail.com> parents: 50893 diff changeset	627 class BuiltinImporter:
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	628
52165 62f9dad628de A few more docstring/API cleanups for importlib. Brett Cannon <bcannon@gmail.com> parents: 52164 diff changeset	629 """Meta path import for built-in modules.
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	630
52165 62f9dad628de A few more docstring/API cleanups for importlib. Brett Cannon <bcannon@gmail.com> parents: 52164 diff changeset	631 All methods are either class or static methods to avoid the need to
62f9dad628de A few more docstring/API cleanups for importlib. Brett Cannon <bcannon@gmail.com> parents: 52164 diff changeset	632 instantiate the class.
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	633
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	634 """
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	635
50983 0940a4f2506e Add importlib.machinery with its first tenants, BuitinImporter and Brett Cannon <bcannon@gmail.com> parents: 50893 diff changeset	636 @classmethod
77126 702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	637 def module_repr(cls, module):
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	638 return "<module '{}' (built-in)>".format(module.__name__)
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	639
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	640 @classmethod
50983 0940a4f2506e Add importlib.machinery with its first tenants, BuitinImporter and Brett Cannon <bcannon@gmail.com> parents: 50893 diff changeset	641 def find_module(cls, fullname, path=None):
52165 62f9dad628de A few more docstring/API cleanups for importlib. Brett Cannon <bcannon@gmail.com> parents: 52164 diff changeset	642 """Find the built-in module.
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	643
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	644 If 'path' is ever specified then the search is considered a failure.
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	645
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	646 """
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	647 if path is not None:
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	648 return None
76331 d777f854a66e Issue #13959: Rename imp to _imp and add Lib/imp.py and begin Brett Cannon <brett@python.org> parents: 76330 diff changeset	649 return cls if _imp.is_builtin(fullname) else None
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	650
51026 c26f1af26bfc Document both importlib.machinery.BuiltinImporter and FrozenImporter. Brett Cannon <bcannon@gmail.com> parents: 50983 diff changeset	651 @classmethod
52002 76a579b6bade Rename importlib.util.set___package__ to set_package. Brett Cannon <bcannon@gmail.com> parents: 51995 diff changeset	652 @set_package
52115 afd7118c8ccf Implement importlib.util.set_loader: a decorator to automatically set Brett Cannon <bcannon@gmail.com> parents: 52112 diff changeset	653 @set_loader
52166 69cd615b76fb Implement InspectLoader for BuiltinImporter. Brett Cannon <bcannon@gmail.com> parents: 52165 diff changeset	654 @_requires_builtin
51026 c26f1af26bfc Document both importlib.machinery.BuiltinImporter and FrozenImporter. Brett Cannon <bcannon@gmail.com> parents: 50983 diff changeset	655 def load_module(cls, fullname):
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	656 """Load a built-in module."""
51681 147fcebacb5b Implement the more specific PEP 302 semantics for loaders and what happens upon Brett Cannon <bcannon@gmail.com> parents: 51639 diff changeset	657 is_reload = fullname in sys.modules
147fcebacb5b Implement the more specific PEP 302 semantics for loaders and what happens upon Brett Cannon <bcannon@gmail.com> parents: 51639 diff changeset	658 try:
78344 62033490ca0f Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise Nick Coghlan <ncoghlan@gmail.com> parents: 78314 diff changeset	659 return _call_with_frames_removed(_imp.init_builtin, fullname)
51681 147fcebacb5b Implement the more specific PEP 302 semantics for loaders and what happens upon Brett Cannon <bcannon@gmail.com> parents: 51639 diff changeset	660 except:
147fcebacb5b Implement the more specific PEP 302 semantics for loaders and what happens upon Brett Cannon <bcannon@gmail.com> parents: 51639 diff changeset	661 if not is_reload and fullname in sys.modules:
147fcebacb5b Implement the more specific PEP 302 semantics for loaders and what happens upon Brett Cannon <bcannon@gmail.com> parents: 51639 diff changeset	662 del sys.modules[fullname]
147fcebacb5b Implement the more specific PEP 302 semantics for loaders and what happens upon Brett Cannon <bcannon@gmail.com> parents: 51639 diff changeset	663 raise
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	664
52166 69cd615b76fb Implement InspectLoader for BuiltinImporter. Brett Cannon <bcannon@gmail.com> parents: 52165 diff changeset	665 @classmethod
69cd615b76fb Implement InspectLoader for BuiltinImporter. Brett Cannon <bcannon@gmail.com> parents: 52165 diff changeset	666 @_requires_builtin
69cd615b76fb Implement InspectLoader for BuiltinImporter. Brett Cannon <bcannon@gmail.com> parents: 52165 diff changeset	667 def get_code(cls, fullname):
69cd615b76fb Implement InspectLoader for BuiltinImporter. Brett Cannon <bcannon@gmail.com> parents: 52165 diff changeset	668 """Return None as built-in modules do not have code objects."""
69cd615b76fb Implement InspectLoader for BuiltinImporter. Brett Cannon <bcannon@gmail.com> parents: 52165 diff changeset	669 return None
69cd615b76fb Implement InspectLoader for BuiltinImporter. Brett Cannon <bcannon@gmail.com> parents: 52165 diff changeset	670
69cd615b76fb Implement InspectLoader for BuiltinImporter. Brett Cannon <bcannon@gmail.com> parents: 52165 diff changeset	671 @classmethod
69cd615b76fb Implement InspectLoader for BuiltinImporter. Brett Cannon <bcannon@gmail.com> parents: 52165 diff changeset	672 @_requires_builtin
69cd615b76fb Implement InspectLoader for BuiltinImporter. Brett Cannon <bcannon@gmail.com> parents: 52165 diff changeset	673 def get_source(cls, fullname):
69cd615b76fb Implement InspectLoader for BuiltinImporter. Brett Cannon <bcannon@gmail.com> parents: 52165 diff changeset	674 """Return None as built-in modules do not have source code."""
69cd615b76fb Implement InspectLoader for BuiltinImporter. Brett Cannon <bcannon@gmail.com> parents: 52165 diff changeset	675 return None
69cd615b76fb Implement InspectLoader for BuiltinImporter. Brett Cannon <bcannon@gmail.com> parents: 52165 diff changeset	676
69cd615b76fb Implement InspectLoader for BuiltinImporter. Brett Cannon <bcannon@gmail.com> parents: 52165 diff changeset	677 @classmethod
69cd615b76fb Implement InspectLoader for BuiltinImporter. Brett Cannon <bcannon@gmail.com> parents: 52165 diff changeset	678 @_requires_builtin
69cd615b76fb Implement InspectLoader for BuiltinImporter. Brett Cannon <bcannon@gmail.com> parents: 52165 diff changeset	679 def is_package(cls, fullname):
77964 37850f083af5 Minor refactoring in importlib._bootstrap, and fix the '_wrap' docstring. Florent Xicluna <florent.xicluna@gmail.com> parents: 77917 diff changeset	680 """Return False as built-in modules are never packages."""
52166 69cd615b76fb Implement InspectLoader for BuiltinImporter. Brett Cannon <bcannon@gmail.com> parents: 52165 diff changeset	681 return False
69cd615b76fb Implement InspectLoader for BuiltinImporter. Brett Cannon <bcannon@gmail.com> parents: 52165 diff changeset	682
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	683
50983 0940a4f2506e Add importlib.machinery with its first tenants, BuitinImporter and Brett Cannon <bcannon@gmail.com> parents: 50893 diff changeset	684 class FrozenImporter:
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	685
52165 62f9dad628de A few more docstring/API cleanups for importlib. Brett Cannon <bcannon@gmail.com> parents: 52164 diff changeset	686 """Meta path import for frozen modules.
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	687
52165 62f9dad628de A few more docstring/API cleanups for importlib. Brett Cannon <bcannon@gmail.com> parents: 52164 diff changeset	688 All methods are either class or static methods to avoid the need to
62f9dad628de A few more docstring/API cleanups for importlib. Brett Cannon <bcannon@gmail.com> parents: 52164 diff changeset	689 instantiate the class.
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	690
50983 0940a4f2506e Add importlib.machinery with its first tenants, BuitinImporter and Brett Cannon <bcannon@gmail.com> parents: 50893 diff changeset	691 """
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	692
50983 0940a4f2506e Add importlib.machinery with its first tenants, BuitinImporter and Brett Cannon <bcannon@gmail.com> parents: 50893 diff changeset	693 @classmethod
77126 702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	694 def module_repr(cls, m):
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	695 return "<module '{}' (frozen)>".format(m.__name__)
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	696
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	697 @classmethod
50983 0940a4f2506e Add importlib.machinery with its first tenants, BuitinImporter and Brett Cannon <bcannon@gmail.com> parents: 50893 diff changeset	698 def find_module(cls, fullname, path=None):
0940a4f2506e Add importlib.machinery with its first tenants, BuitinImporter and Brett Cannon <bcannon@gmail.com> parents: 50893 diff changeset	699 """Find a frozen module."""
76331 d777f854a66e Issue #13959: Rename imp to _imp and add Lib/imp.py and begin Brett Cannon <brett@python.org> parents: 76330 diff changeset	700 return cls if _imp.is_frozen(fullname) else None
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	701
50983 0940a4f2506e Add importlib.machinery with its first tenants, BuitinImporter and Brett Cannon <bcannon@gmail.com> parents: 50893 diff changeset	702 @classmethod
52002 76a579b6bade Rename importlib.util.set___package__ to set_package. Brett Cannon <bcannon@gmail.com> parents: 51995 diff changeset	703 @set_package
52115 afd7118c8ccf Implement importlib.util.set_loader: a decorator to automatically set Brett Cannon <bcannon@gmail.com> parents: 52112 diff changeset	704 @set_loader
52168 70f5f9113ae1 Implement InspectLoader for FrozenImporter. Brett Cannon <bcannon@gmail.com> parents: 52166 diff changeset	705 @_requires_frozen
50983 0940a4f2506e Add importlib.machinery with its first tenants, BuitinImporter and Brett Cannon <bcannon@gmail.com> parents: 50893 diff changeset	706 def load_module(cls, fullname):
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	707 """Load a frozen module."""
51681 147fcebacb5b Implement the more specific PEP 302 semantics for loaders and what happens upon Brett Cannon <bcannon@gmail.com> parents: 51639 diff changeset	708 is_reload = fullname in sys.modules
147fcebacb5b Implement the more specific PEP 302 semantics for loaders and what happens upon Brett Cannon <bcannon@gmail.com> parents: 51639 diff changeset	709 try:
78344 62033490ca0f Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise Nick Coghlan <ncoghlan@gmail.com> parents: 78314 diff changeset	710 m = _call_with_frames_removed(_imp.init_frozen, fullname)
77126 702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	711 # Let our own module_repr() method produce a suitable repr.
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	712 del m.__file__
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	713 return m
51681 147fcebacb5b Implement the more specific PEP 302 semantics for loaders and what happens upon Brett Cannon <bcannon@gmail.com> parents: 51639 diff changeset	714 except:
147fcebacb5b Implement the more specific PEP 302 semantics for loaders and what happens upon Brett Cannon <bcannon@gmail.com> parents: 51639 diff changeset	715 if not is_reload and fullname in sys.modules:
147fcebacb5b Implement the more specific PEP 302 semantics for loaders and what happens upon Brett Cannon <bcannon@gmail.com> parents: 51639 diff changeset	716 del sys.modules[fullname]
147fcebacb5b Implement the more specific PEP 302 semantics for loaders and what happens upon Brett Cannon <bcannon@gmail.com> parents: 51639 diff changeset	717 raise
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	718
52168 70f5f9113ae1 Implement InspectLoader for FrozenImporter. Brett Cannon <bcannon@gmail.com> parents: 52166 diff changeset	719 @classmethod
70f5f9113ae1 Implement InspectLoader for FrozenImporter. Brett Cannon <bcannon@gmail.com> parents: 52166 diff changeset	720 @_requires_frozen
70f5f9113ae1 Implement InspectLoader for FrozenImporter. Brett Cannon <bcannon@gmail.com> parents: 52166 diff changeset	721 def get_code(cls, fullname):
70f5f9113ae1 Implement InspectLoader for FrozenImporter. Brett Cannon <bcannon@gmail.com> parents: 52166 diff changeset	722 """Return the code object for the frozen module."""
76331 d777f854a66e Issue #13959: Rename imp to _imp and add Lib/imp.py and begin Brett Cannon <brett@python.org> parents: 76330 diff changeset	723 return _imp.get_frozen_object(fullname)
52168 70f5f9113ae1 Implement InspectLoader for FrozenImporter. Brett Cannon <bcannon@gmail.com> parents: 52166 diff changeset	724
70f5f9113ae1 Implement InspectLoader for FrozenImporter. Brett Cannon <bcannon@gmail.com> parents: 52166 diff changeset	725 @classmethod
70f5f9113ae1 Implement InspectLoader for FrozenImporter. Brett Cannon <bcannon@gmail.com> parents: 52166 diff changeset	726 @_requires_frozen
70f5f9113ae1 Implement InspectLoader for FrozenImporter. Brett Cannon <bcannon@gmail.com> parents: 52166 diff changeset	727 def get_source(cls, fullname):
70f5f9113ae1 Implement InspectLoader for FrozenImporter. Brett Cannon <bcannon@gmail.com> parents: 52166 diff changeset	728 """Return None as frozen modules do not have source code."""
70f5f9113ae1 Implement InspectLoader for FrozenImporter. Brett Cannon <bcannon@gmail.com> parents: 52166 diff changeset	729 return None
70f5f9113ae1 Implement InspectLoader for FrozenImporter. Brett Cannon <bcannon@gmail.com> parents: 52166 diff changeset	730
70f5f9113ae1 Implement InspectLoader for FrozenImporter. Brett Cannon <bcannon@gmail.com> parents: 52166 diff changeset	731 @classmethod
70f5f9113ae1 Implement InspectLoader for FrozenImporter. Brett Cannon <bcannon@gmail.com> parents: 52166 diff changeset	732 @_requires_frozen
70f5f9113ae1 Implement InspectLoader for FrozenImporter. Brett Cannon <bcannon@gmail.com> parents: 52166 diff changeset	733 def is_package(cls, fullname):
70f5f9113ae1 Implement InspectLoader for FrozenImporter. Brett Cannon <bcannon@gmail.com> parents: 52166 diff changeset	734 """Return if the frozen module is a package."""
76331 d777f854a66e Issue #13959: Rename imp to _imp and add Lib/imp.py and begin Brett Cannon <brett@python.org> parents: 76330 diff changeset	735 return _imp.is_frozen_package(fullname)
52168 70f5f9113ae1 Implement InspectLoader for FrozenImporter. Brett Cannon <bcannon@gmail.com> parents: 52166 diff changeset	736
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	737
78380 a1ac1e13c5a0 Close #15519: Properly expose WindowsRegistryFinder in importlib and bring the name into line with normal import terminology. Original patch by Eric Snow Nick Coghlan <ncoghlan@gmail.com> parents: 78344 diff changeset	738 class WindowsRegistryFinder:
78308 bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	739
78380 a1ac1e13c5a0 Close #15519: Properly expose WindowsRegistryFinder in importlib and bring the name into line with normal import terminology. Original patch by Eric Snow Nick Coghlan <ncoghlan@gmail.com> parents: 78344 diff changeset	740 """Meta path finder for modules declared in the Windows registry.
78308 bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	741 """
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	742
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	743 REGISTRY_KEY = (
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	744 "Software\\Python\\PythonCore\\{sys_version}"
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	745 "\\Modules\\{fullname}")
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	746 REGISTRY_KEY_DEBUG = (
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	747 "Software\\Python\\PythonCore\\{sys_version}"
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	748 "\\Modules\\{fullname}\\Debug")
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	749 DEBUG_BUILD = False # Changed in _setup()
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	750
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	751 @classmethod
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	752 def _open_registry(cls, key):
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	753 try:
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	754 return _winreg.OpenKey(_winreg.HKEY_CURRENT_USER, key)
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	755 except WindowsError:
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	756 return _winreg.OpenKey(_winreg.HKEY_LOCAL_MACHINE, key)
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	757
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	758 @classmethod
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	759 def _search_registry(cls, fullname):
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	760 if cls.DEBUG_BUILD:
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	761 registry_key = cls.REGISTRY_KEY_DEBUG
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	762 else:
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	763 registry_key = cls.REGISTRY_KEY
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	764 key = registry_key.format(fullname=fullname,
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	765 sys_version=sys.version[:3])
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	766 try:
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	767 with cls._open_registry(key) as hkey:
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	768 filepath = _winreg.QueryValue(hkey, "")
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	769 except WindowsError:
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	770 return None
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	771 return filepath
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	772
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	773 @classmethod
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	774 def find_module(cls, fullname, path=None):
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	775 """Find module named in the registry."""
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	776 filepath = cls._search_registry(fullname)
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	777 if filepath is None:
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	778 return None
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	779 try:
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	780 _os.stat(filepath)
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	781 except OSError:
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	782 return None
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	783 for loader, suffixes, _ in _get_supported_file_loaders():
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	784 if filepath.endswith(tuple(suffixes)):
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	785 return loader(fullname, filepath)
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	786
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	787
62756 34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	788 class _LoaderBasics:
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	789
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	790 """Base class of common code needed by both SourceLoader and
76478 1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and Brett Cannon <brett@python.org> parents: 76472 diff changeset	791 SourcelessFileLoader."""
51771 ee7029710c05 Refactor source and bytecode file loaders in importlib so that there Brett Cannon <bcannon@gmail.com> parents: 51770 diff changeset	792
62756 34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	793 def is_package(self, fullname):
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	794 """Concrete implementation of InspectLoader.is_package by checking if
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	795 the path returned by get_filename has a filename of '__init__.py'."""
76449 163360370ad2 Make path manipulation more robust for platforms with alternative path Brett Cannon <brett@python.org> parents: 76445 diff changeset	796 filename = _path_split(self.get_filename(fullname))[1]
77457 240b7467e65c Issue #14938: importlib.abc.SourceLoader.is_package() now takes the Brett Cannon <brett@python.org> parents: 77180 diff changeset	797 filename_base = filename.rsplit('.', 1)[0]
240b7467e65c Issue #14938: importlib.abc.SourceLoader.is_package() now takes the Brett Cannon <brett@python.org> parents: 77180 diff changeset	798 tail_name = fullname.rpartition('.')[2]
240b7467e65c Issue #14938: importlib.abc.SourceLoader.is_package() now takes the Brett Cannon <brett@python.org> parents: 77180 diff changeset	799 return filename_base == '__init__' and tail_name != '__init__'
52165 62f9dad628de A few more docstring/API cleanups for importlib. Brett Cannon <bcannon@gmail.com> parents: 52164 diff changeset	800
76280 c071af95772a Have importlib take advantage of ImportError's new 'name' and 'path' Brett Cannon <brett@python.org> parents: 76081 diff changeset	801 def _bytes_from_bytecode(self, fullname, data, bytecode_path, source_stats):
62756 34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	802 """Return the marshalled bytes from bytecode, verifying the magic
74358 87331661042b Issue #13645: pyc files now contain the size of the corresponding source Antoine Pitrou <solipsis@pitrou.net> parents: 73977 diff changeset	803 number, timestamp and source size along the way.
51771 ee7029710c05 Refactor source and bytecode file loaders in importlib so that there Brett Cannon <bcannon@gmail.com> parents: 51770 diff changeset	804
74358 87331661042b Issue #13645: pyc files now contain the size of the corresponding source Antoine Pitrou <solipsis@pitrou.net> parents: 73977 diff changeset	805 If source_stats is None then skip the timestamp check.
51771 ee7029710c05 Refactor source and bytecode file loaders in importlib so that there Brett Cannon <bcannon@gmail.com> parents: 51770 diff changeset	806
62756 34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	807 """
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	808 magic = data[:4]
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	809 raw_timestamp = data[4:8]
74358 87331661042b Issue #13645: pyc files now contain the size of the corresponding source Antoine Pitrou <solipsis@pitrou.net> parents: 73977 diff changeset	810 raw_size = data[8:12]
78042 efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	811 if magic != _MAGIC_BYTES:
76445 c820aa9c0c00 Issue #13959: Keep imp.get_magic() in C code, but cache in importlib Brett Cannon <brett@python.org> parents: 76428 diff changeset	812 msg = 'bad magic number in {!r}: {!r}'.format(fullname, magic)
c820aa9c0c00 Issue #13959: Keep imp.get_magic() in C code, but cache in importlib Brett Cannon <brett@python.org> parents: 76428 diff changeset	813 raise ImportError(msg, name=fullname, path=bytecode_path)
62756 34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	814 elif len(raw_timestamp) != 4:
76303 2dd046be2c88 Issue #2377: Make importlib the implementation of __import__(). Brett Cannon <brett@python.org> parents: 76280 diff changeset	815 message = 'bad timestamp in {}'.format(fullname)
76753 59b8a7fcb047 Properly mark names in importlib._bootstrap as private. Brett Cannon <brett@python.org> parents: 76586 diff changeset	816 _verbose_message(message)
76303 2dd046be2c88 Issue #2377: Make importlib the implementation of __import__(). Brett Cannon <brett@python.org> parents: 76280 diff changeset	817 raise EOFError(message)
74358 87331661042b Issue #13645: pyc files now contain the size of the corresponding source Antoine Pitrou <solipsis@pitrou.net> parents: 73977 diff changeset	818 elif len(raw_size) != 4:
76303 2dd046be2c88 Issue #2377: Make importlib the implementation of __import__(). Brett Cannon <brett@python.org> parents: 76280 diff changeset	819 message = 'bad size in {}'.format(fullname)
76753 59b8a7fcb047 Properly mark names in importlib._bootstrap as private. Brett Cannon <brett@python.org> parents: 76586 diff changeset	820 _verbose_message(message)
76303 2dd046be2c88 Issue #2377: Make importlib the implementation of __import__(). Brett Cannon <brett@python.org> parents: 76280 diff changeset	821 raise EOFError(message)
74358 87331661042b Issue #13645: pyc files now contain the size of the corresponding source Antoine Pitrou <solipsis@pitrou.net> parents: 73977 diff changeset	822 if source_stats is not None:
87331661042b Issue #13645: pyc files now contain the size of the corresponding source Antoine Pitrou <solipsis@pitrou.net> parents: 73977 diff changeset	823 try:
87331661042b Issue #13645: pyc files now contain the size of the corresponding source Antoine Pitrou <solipsis@pitrou.net> parents: 73977 diff changeset	824 source_mtime = int(source_stats['mtime'])
87331661042b Issue #13645: pyc files now contain the size of the corresponding source Antoine Pitrou <solipsis@pitrou.net> parents: 73977 diff changeset	825 except KeyError:
87331661042b Issue #13645: pyc files now contain the size of the corresponding source Antoine Pitrou <solipsis@pitrou.net> parents: 73977 diff changeset	826 pass
87331661042b Issue #13645: pyc files now contain the size of the corresponding source Antoine Pitrou <solipsis@pitrou.net> parents: 73977 diff changeset	827 else:
74630 7d8f016784cd Move some code from importlib.__init__ to importlib._bootstrap that Brett Cannon <brett@python.org> parents: 74449 diff changeset	828 if _r_long(raw_timestamp) != source_mtime:
76303 2dd046be2c88 Issue #2377: Make importlib the implementation of __import__(). Brett Cannon <brett@python.org> parents: 76280 diff changeset	829 message = 'bytecode is stale for {}'.format(fullname)
76753 59b8a7fcb047 Properly mark names in importlib._bootstrap as private. Brett Cannon <brett@python.org> parents: 76586 diff changeset	830 _verbose_message(message)
76303 2dd046be2c88 Issue #2377: Make importlib the implementation of __import__(). Brett Cannon <brett@python.org> parents: 76280 diff changeset	831 raise ImportError(message, name=fullname,
2dd046be2c88 Issue #2377: Make importlib the implementation of __import__(). Brett Cannon <brett@python.org> parents: 76280 diff changeset	832 path=bytecode_path)
74358 87331661042b Issue #13645: pyc files now contain the size of the corresponding source Antoine Pitrou <solipsis@pitrou.net> parents: 73977 diff changeset	833 try:
87331661042b Issue #13645: pyc files now contain the size of the corresponding source Antoine Pitrou <solipsis@pitrou.net> parents: 73977 diff changeset	834 source_size = source_stats['size'] & 0xFFFFFFFF
87331661042b Issue #13645: pyc files now contain the size of the corresponding source Antoine Pitrou <solipsis@pitrou.net> parents: 73977 diff changeset	835 except KeyError:
87331661042b Issue #13645: pyc files now contain the size of the corresponding source Antoine Pitrou <solipsis@pitrou.net> parents: 73977 diff changeset	836 pass
87331661042b Issue #13645: pyc files now contain the size of the corresponding source Antoine Pitrou <solipsis@pitrou.net> parents: 73977 diff changeset	837 else:
74630 7d8f016784cd Move some code from importlib.__init__ to importlib._bootstrap that Brett Cannon <brett@python.org> parents: 74449 diff changeset	838 if _r_long(raw_size) != source_size:
75250 8840037a9c53 unused imports, pep8 Philip Jenvey <pjenvey@underboss.org> parents: 75239 diff changeset	839 raise ImportError(
76280 c071af95772a Have importlib take advantage of ImportError's new 'name' and 'path' Brett Cannon <brett@python.org> parents: 76081 diff changeset	840 "bytecode is stale for {}".format(fullname),
c071af95772a Have importlib take advantage of ImportError's new 'name' and 'path' Brett Cannon <brett@python.org> parents: 76081 diff changeset	841 name=fullname, path=bytecode_path)
62756 34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	842 # Can't return the code object as errors from marshal loading need to
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	843 # propagate even when source is available.
74358 87331661042b Issue #13645: pyc files now contain the size of the corresponding source Antoine Pitrou <solipsis@pitrou.net> parents: 73977 diff changeset	844 return data[12:]
51771 ee7029710c05 Refactor source and bytecode file loaders in importlib so that there Brett Cannon <bcannon@gmail.com> parents: 51770 diff changeset	845
ee7029710c05 Refactor source and bytecode file loaders in importlib so that there Brett Cannon <bcannon@gmail.com> parents: 51770 diff changeset	846 @module_for_loader
62756 34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	847 def _load_module(self, module, *, sourceless=False):
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	848 """Helper for load_module able to handle either source or sourceless
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	849 loading."""
51771 ee7029710c05 Refactor source and bytecode file loaders in importlib so that there Brett Cannon <bcannon@gmail.com> parents: 51770 diff changeset	850 name = module.__name__
62756 34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	851 code_object = self.get_code(name)
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	852 module.__file__ = self.get_filename(name)
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	853 if not sourceless:
78026 e86330669bb5 Issue #15056: imp.cache_from_source() and source_from_cache() raise Brett Cannon <brett@python.org> parents: 78010 diff changeset	854 try:
e86330669bb5 Issue #15056: imp.cache_from_source() and source_from_cache() raise Brett Cannon <brett@python.org> parents: 78010 diff changeset	855 module.__cached__ = cache_from_source(module.__file__)
e86330669bb5 Issue #15056: imp.cache_from_source() and source_from_cache() raise Brett Cannon <brett@python.org> parents: 78010 diff changeset	856 except NotImplementedError:
e86330669bb5 Issue #15056: imp.cache_from_source() and source_from_cache() raise Brett Cannon <brett@python.org> parents: 78010 diff changeset	857 module.__cached__ = module.__file__
62756 34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	858 else:
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	859 module.__cached__ = module.__file__
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	860 module.__package__ = name
51771 ee7029710c05 Refactor source and bytecode file loaders in importlib so that there Brett Cannon <bcannon@gmail.com> parents: 51770 diff changeset	861 if self.is_package(name):
76449 163360370ad2 Make path manipulation more robust for platforms with alternative path Brett Cannon <brett@python.org> parents: 76445 diff changeset	862 module.__path__ = [_path_split(module.__file__)[0]]
62756 34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	863 else:
51771 ee7029710c05 Refactor source and bytecode file loaders in importlib so that there Brett Cannon <bcannon@gmail.com> parents: 51770 diff changeset	864 module.__package__ = module.__package__.rpartition('.')[0]
52075 2033d45a5b99 Minor changes to Python source base loader. Brett Cannon <bcannon@gmail.com> parents: 52002 diff changeset	865 module.__loader__ = self
78344 62033490ca0f Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise Nick Coghlan <ncoghlan@gmail.com> parents: 78314 diff changeset	866 _call_with_frames_removed(exec, code_object, module.__dict__)
51771 ee7029710c05 Refactor source and bytecode file loaders in importlib so that there Brett Cannon <bcannon@gmail.com> parents: 51770 diff changeset	867 return module
ee7029710c05 Refactor source and bytecode file loaders in importlib so that there Brett Cannon <bcannon@gmail.com> parents: 51770 diff changeset	868
62756 34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	869
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	870 class SourceLoader(_LoaderBasics):
51771 ee7029710c05 Refactor source and bytecode file loaders in importlib so that there Brett Cannon <bcannon@gmail.com> parents: 51770 diff changeset	871
67573 e125d92f7438 Issue 10899: Remove function type annotations from the stdlib Raymond Hettinger <python@rcn.com> parents: 66226 diff changeset	872 def path_mtime(self, path):
67581 7d658b49b3e1 Issue 10899: Remove function type annotations from the stdlib Raymond Hettinger <python@rcn.com> parents: 67573 diff changeset	873 """Optional method that returns the modification time (an int) for the
7d658b49b3e1 Issue 10899: Remove function type annotations from the stdlib Raymond Hettinger <python@rcn.com> parents: 67573 diff changeset	874 specified path, where path is a str.
74358 87331661042b Issue #13645: pyc files now contain the size of the corresponding source Antoine Pitrou <solipsis@pitrou.net> parents: 73977 diff changeset	875 """
87331661042b Issue #13645: pyc files now contain the size of the corresponding source Antoine Pitrou <solipsis@pitrou.net> parents: 73977 diff changeset	876 raise NotImplementedError
87331661042b Issue #13645: pyc files now contain the size of the corresponding source Antoine Pitrou <solipsis@pitrou.net> parents: 73977 diff changeset	877
87331661042b Issue #13645: pyc files now contain the size of the corresponding source Antoine Pitrou <solipsis@pitrou.net> parents: 73977 diff changeset	878 def path_stats(self, path):
87331661042b Issue #13645: pyc files now contain the size of the corresponding source Antoine Pitrou <solipsis@pitrou.net> parents: 73977 diff changeset	879 """Optional method returning a metadata dict for the specified path
87331661042b Issue #13645: pyc files now contain the size of the corresponding source Antoine Pitrou <solipsis@pitrou.net> parents: 73977 diff changeset	880 to by the path (str).
87331661042b Issue #13645: pyc files now contain the size of the corresponding source Antoine Pitrou <solipsis@pitrou.net> parents: 73977 diff changeset	881 Possible keys:
87331661042b Issue #13645: pyc files now contain the size of the corresponding source Antoine Pitrou <solipsis@pitrou.net> parents: 73977 diff changeset	882 - 'mtime' (mandatory) is the numeric timestamp of last source
87331661042b Issue #13645: pyc files now contain the size of the corresponding source Antoine Pitrou <solipsis@pitrou.net> parents: 73977 diff changeset	883 code modification;
87331661042b Issue #13645: pyc files now contain the size of the corresponding source Antoine Pitrou <solipsis@pitrou.net> parents: 73977 diff changeset	884 - 'size' (optional) is the size in bytes of the source code.
52112 03080fa08dfe Implement get_source for importlib.abc.PyLoader using source_path and get_data. Brett Cannon <bcannon@gmail.com> parents: 52083 diff changeset	885
62587 b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	886 Implementing this method allows the loader to read bytecode files.
52112 03080fa08dfe Implement get_source for importlib.abc.PyLoader using source_path and get_data. Brett Cannon <bcannon@gmail.com> parents: 52083 diff changeset	887 """
74358 87331661042b Issue #13645: pyc files now contain the size of the corresponding source Antoine Pitrou <solipsis@pitrou.net> parents: 73977 diff changeset	888 return {'mtime': self.path_mtime(path)}
62587 b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	889
67573 e125d92f7438 Issue 10899: Remove function type annotations from the stdlib Raymond Hettinger <python@rcn.com> parents: 66226 diff changeset	890 def set_data(self, path, data):
67581 7d658b49b3e1 Issue 10899: Remove function type annotations from the stdlib Raymond Hettinger <python@rcn.com> parents: 67573 diff changeset	891 """Optional method which writes data (bytes) to a file path (a str).
62587 b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	892
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	893 Implementing this method allows for the writing of bytecode files.
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	894
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	895 """
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	896 raise NotImplementedError
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	897
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	898
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	899 def get_source(self, fullname):
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	900 """Concrete implementation of InspectLoader.get_source."""
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	901 import tokenize
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	902 path = self.get_filename(fullname)
52112 03080fa08dfe Implement get_source for importlib.abc.PyLoader using source_path and get_data. Brett Cannon <bcannon@gmail.com> parents: 52083 diff changeset	903 try:
03080fa08dfe Implement get_source for importlib.abc.PyLoader using source_path and get_data. Brett Cannon <bcannon@gmail.com> parents: 52083 diff changeset	904 source_bytes = self.get_data(path)
78112 7d202353a728 Issue #15343: A lot more than just unicode decoding can go wrong when retrieving a source file Nick Coghlan <ncoghlan@gmail.com> parents: 78078 diff changeset	905 except IOError as exc:
76280 c071af95772a Have importlib take advantage of ImportError's new 'name' and 'path' Brett Cannon <brett@python.org> parents: 76081 diff changeset	906 raise ImportError("source not available through get_data()",
78112 7d202353a728 Issue #15343: A lot more than just unicode decoding can go wrong when retrieving a source file Nick Coghlan <ncoghlan@gmail.com> parents: 78078 diff changeset	907 name=fullname) from exc
7d202353a728 Issue #15343: A lot more than just unicode decoding can go wrong when retrieving a source file Nick Coghlan <ncoghlan@gmail.com> parents: 78078 diff changeset	908 readsource = _io.BytesIO(source_bytes).readline
7d202353a728 Issue #15343: A lot more than just unicode decoding can go wrong when retrieving a source file Nick Coghlan <ncoghlan@gmail.com> parents: 78078 diff changeset	909 try:
7d202353a728 Issue #15343: A lot more than just unicode decoding can go wrong when retrieving a source file Nick Coghlan <ncoghlan@gmail.com> parents: 78078 diff changeset	910 encoding = tokenize.detect_encoding(readsource)
7d202353a728 Issue #15343: A lot more than just unicode decoding can go wrong when retrieving a source file Nick Coghlan <ncoghlan@gmail.com> parents: 78078 diff changeset	911 except SyntaxError as exc:
7d202353a728 Issue #15343: A lot more than just unicode decoding can go wrong when retrieving a source file Nick Coghlan <ncoghlan@gmail.com> parents: 78078 diff changeset	912 raise ImportError("Failed to detect encoding",
7d202353a728 Issue #15343: A lot more than just unicode decoding can go wrong when retrieving a source file Nick Coghlan <ncoghlan@gmail.com> parents: 78078 diff changeset	913 name=fullname) from exc
62761 32682b253ffc Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62756 diff changeset	914 newline_decoder = _io.IncrementalNewlineDecoder(None, True)
78112 7d202353a728 Issue #15343: A lot more than just unicode decoding can go wrong when retrieving a source file Nick Coghlan <ncoghlan@gmail.com> parents: 78078 diff changeset	915 try:
7d202353a728 Issue #15343: A lot more than just unicode decoding can go wrong when retrieving a source file Nick Coghlan <ncoghlan@gmail.com> parents: 78078 diff changeset	916 return newline_decoder.decode(source_bytes.decode(encoding[0]))
7d202353a728 Issue #15343: A lot more than just unicode decoding can go wrong when retrieving a source file Nick Coghlan <ncoghlan@gmail.com> parents: 78078 diff changeset	917 except UnicodeDecodeError as exc:
7d202353a728 Issue #15343: A lot more than just unicode decoding can go wrong when retrieving a source file Nick Coghlan <ncoghlan@gmail.com> parents: 78078 diff changeset	918 raise ImportError("Failed to decode source file",
7d202353a728 Issue #15343: A lot more than just unicode decoding can go wrong when retrieving a source file Nick Coghlan <ncoghlan@gmail.com> parents: 78078 diff changeset	919 name=fullname) from exc
51771 ee7029710c05 Refactor source and bytecode file loaders in importlib so that there Brett Cannon <bcannon@gmail.com> parents: 51770 diff changeset	920
ee7029710c05 Refactor source and bytecode file loaders in importlib so that there Brett Cannon <bcannon@gmail.com> parents: 51770 diff changeset	921 def get_code(self, fullname):
62587 b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	922 """Concrete implementation of InspectLoader.get_code.
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	923
74358 87331661042b Issue #13645: pyc files now contain the size of the corresponding source Antoine Pitrou <solipsis@pitrou.net> parents: 73977 diff changeset	924 Reading of bytecode requires path_stats to be implemented. To write
62587 b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	925 bytecode, set_data must also be implemented.
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	926
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	927 """
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	928 source_path = self.get_filename(fullname)
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	929 source_mtime = None
78026 e86330669bb5 Issue #15056: imp.cache_from_source() and source_from_cache() raise Brett Cannon <brett@python.org> parents: 78010 diff changeset	930 try:
e86330669bb5 Issue #15056: imp.cache_from_source() and source_from_cache() raise Brett Cannon <brett@python.org> parents: 78010 diff changeset	931 bytecode_path = cache_from_source(source_path)
e86330669bb5 Issue #15056: imp.cache_from_source() and source_from_cache() raise Brett Cannon <brett@python.org> parents: 78010 diff changeset	932 except NotImplementedError:
e86330669bb5 Issue #15056: imp.cache_from_source() and source_from_cache() raise Brett Cannon <brett@python.org> parents: 78010 diff changeset	933 bytecode_path = None
e86330669bb5 Issue #15056: imp.cache_from_source() and source_from_cache() raise Brett Cannon <brett@python.org> parents: 78010 diff changeset	934 else:
51771 ee7029710c05 Refactor source and bytecode file loaders in importlib so that there Brett Cannon <bcannon@gmail.com> parents: 51770 diff changeset	935 try:
74358 87331661042b Issue #13645: pyc files now contain the size of the corresponding source Antoine Pitrou <solipsis@pitrou.net> parents: 73977 diff changeset	936 st = self.path_stats(source_path)
62587 b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	937 except NotImplementedError:
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	938 pass
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	939 else:
74358 87331661042b Issue #13645: pyc files now contain the size of the corresponding source Antoine Pitrou <solipsis@pitrou.net> parents: 73977 diff changeset	940 source_mtime = int(st['mtime'])
62587 b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	941 try:
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	942 data = self.get_data(bytecode_path)
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	943 except IOError:
51771 ee7029710c05 Refactor source and bytecode file loaders in importlib so that there Brett Cannon <bcannon@gmail.com> parents: 51770 diff changeset	944 pass
ee7029710c05 Refactor source and bytecode file loaders in importlib so that there Brett Cannon <bcannon@gmail.com> parents: 51770 diff changeset	945 else:
62756 34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	946 try:
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	947 bytes_data = self._bytes_from_bytecode(fullname, data,
76280 c071af95772a Have importlib take advantage of ImportError's new 'name' and 'path' Brett Cannon <brett@python.org> parents: 76081 diff changeset	948 bytecode_path,
74358 87331661042b Issue #13645: pyc files now contain the size of the corresponding source Antoine Pitrou <solipsis@pitrou.net> parents: 73977 diff changeset	949 st)
62756 34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	950 except (ImportError, EOFError):
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	951 pass
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	952 else:
76753 59b8a7fcb047 Properly mark names in importlib._bootstrap as private. Brett Cannon <brett@python.org> parents: 76586 diff changeset	953 _verbose_message('{} matches {}', bytecode_path,
76303 2dd046be2c88 Issue #2377: Make importlib the implementation of __import__(). Brett Cannon <brett@python.org> parents: 76280 diff changeset	954 source_path)
62756 34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	955 found = marshal.loads(bytes_data)
76753 59b8a7fcb047 Properly mark names in importlib._bootstrap as private. Brett Cannon <brett@python.org> parents: 76586 diff changeset	956 if isinstance(found, _code_type):
76331 d777f854a66e Issue #13959: Rename imp to _imp and add Lib/imp.py and begin Brett Cannon <brett@python.org> parents: 76330 diff changeset	957 _imp._fix_co_filename(found, source_path)
76753 59b8a7fcb047 Properly mark names in importlib._bootstrap as private. Brett Cannon <brett@python.org> parents: 76586 diff changeset	958 _verbose_message('code object from {}',
76303 2dd046be2c88 Issue #2377: Make importlib the implementation of __import__(). Brett Cannon <brett@python.org> parents: 76280 diff changeset	959 bytecode_path)
62756 34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	960 return found
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	961 else:
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	962 msg = "Non-code object in {}"
76280 c071af95772a Have importlib take advantage of ImportError's new 'name' and 'path' Brett Cannon <brett@python.org> parents: 76081 diff changeset	963 raise ImportError(msg.format(bytecode_path),
c071af95772a Have importlib take advantage of ImportError's new 'name' and 'path' Brett Cannon <brett@python.org> parents: 76081 diff changeset	964 name=fullname, path=bytecode_path)
62587 b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	965 source_bytes = self.get_data(source_path)
78344 62033490ca0f Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise Nick Coghlan <ncoghlan@gmail.com> parents: 78314 diff changeset	966 code_object = _call_with_frames_removed(compile,
62033490ca0f Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise Nick Coghlan <ncoghlan@gmail.com> parents: 78314 diff changeset	967 source_bytes, source_path, 'exec',
62033490ca0f Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise Nick Coghlan <ncoghlan@gmail.com> parents: 78314 diff changeset	968 dont_inherit=True)
76753 59b8a7fcb047 Properly mark names in importlib._bootstrap as private. Brett Cannon <brett@python.org> parents: 76586 diff changeset	969 _verbose_message('code object from {}', source_path)
62587 b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	970 if (not sys.dont_write_bytecode and bytecode_path is not None and
75250 8840037a9c53 unused imports, pep8 Philip Jenvey <pjenvey@underboss.org> parents: 75239 diff changeset	971 source_mtime is not None):
78042 efb5e6ab10f4 Issue #15167 (as part of #13959): imp.get_magic() is no implemented in Brett Cannon <brett@python.org> parents: 78026 diff changeset	972 data = bytearray(_MAGIC_BYTES)
74630 7d8f016784cd Move some code from importlib.__init__ to importlib._bootstrap that Brett Cannon <brett@python.org> parents: 74449 diff changeset	973 data.extend(_w_long(source_mtime))
7d8f016784cd Move some code from importlib.__init__ to importlib._bootstrap that Brett Cannon <brett@python.org> parents: 74449 diff changeset	974 data.extend(_w_long(len(source_bytes)))
51771 ee7029710c05 Refactor source and bytecode file loaders in importlib so that there Brett Cannon <bcannon@gmail.com> parents: 51770 diff changeset	975 data.extend(marshal.dumps(code_object))
62587 b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	976 try:
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	977 self.set_data(bytecode_path, data)
76753 59b8a7fcb047 Properly mark names in importlib._bootstrap as private. Brett Cannon <brett@python.org> parents: 76586 diff changeset	978 _verbose_message('wrote {!r}', bytecode_path)
62587 b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	979 except NotImplementedError:
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	980 pass
51771 ee7029710c05 Refactor source and bytecode file loaders in importlib so that there Brett Cannon <bcannon@gmail.com> parents: 51770 diff changeset	981 return code_object
ee7029710c05 Refactor source and bytecode file loaders in importlib so that there Brett Cannon <bcannon@gmail.com> parents: 51770 diff changeset	982
62756 34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	983 def load_module(self, fullname):
62587 b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	984 """Concrete implementation of Loader.load_module.
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	985
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	986 Requires ExecutionLoader.get_filename and ResourceLoader.get_data to be
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	987 implemented to load source code. Use of bytecode is dictated by whether
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	988 get_code uses/writes bytecode.
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	989
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	990 """
62756 34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	991 return self._load_module(fullname)
62587 b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	992
51771 ee7029710c05 Refactor source and bytecode file loaders in importlib so that there Brett Cannon <bcannon@gmail.com> parents: 51770 diff changeset	993
76478 1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and Brett Cannon <brett@python.org> parents: 76472 diff changeset	994 class FileLoader:
51771 ee7029710c05 Refactor source and bytecode file loaders in importlib so that there Brett Cannon <bcannon@gmail.com> parents: 51770 diff changeset	995
62756 34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	996 """Base file loader class which implements the loader protocol methods that
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	997 require file system usage."""
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	998
62587 b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	999 def __init__(self, fullname, path):
62756 34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	1000 """Cache the module name and the path to the file found by the
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	1001 finder."""
76478 1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and Brett Cannon <brett@python.org> parents: 76472 diff changeset	1002 self.name = fullname
1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and Brett Cannon <brett@python.org> parents: 76472 diff changeset	1003 self.path = path
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	1004
52139 2e8e897a1244 Make utility code in importlib._bootstrap private. Brett Cannon <bcannon@gmail.com> parents: 52115 diff changeset	1005 @_check_name
76877 626d5c6fbd95 Issue #13959: Have Brett Cannon <brett@python.org> parents: 76874 diff changeset	1006 def load_module(self, fullname):
626d5c6fbd95 Issue #13959: Have Brett Cannon <brett@python.org> parents: 76874 diff changeset	1007 """Load a module from a file."""
77180 bcb3b81853cc Tweak importlib._bootstrap to avoid zero-argument super so I can work on issue #14857 without breaking imports Nick Coghlan <ncoghlan@gmail.com> parents: 77164 diff changeset	1008 # Issue #14857: Avoid the zero-argument form so the implementation
bcb3b81853cc Tweak importlib._bootstrap to avoid zero-argument super so I can work on issue #14857 without breaking imports Nick Coghlan <ncoghlan@gmail.com> parents: 77164 diff changeset	1009 # of that form can be updated without breaking the frozen module
bcb3b81853cc Tweak importlib._bootstrap to avoid zero-argument super so I can work on issue #14857 without breaking imports Nick Coghlan <ncoghlan@gmail.com> parents: 77164 diff changeset	1010 return super(FileLoader, self).load_module(fullname)
76877 626d5c6fbd95 Issue #13959: Have Brett Cannon <brett@python.org> parents: 76874 diff changeset	1011
626d5c6fbd95 Issue #13959: Have Brett Cannon <brett@python.org> parents: 76874 diff changeset	1012 @_check_name
62587 b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	1013 def get_filename(self, fullname):
b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	1014 """Return the path to the source file as found by the finder."""
76478 1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and Brett Cannon <brett@python.org> parents: 76472 diff changeset	1015 return self.path
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	1016
51771 ee7029710c05 Refactor source and bytecode file loaders in importlib so that there Brett Cannon <bcannon@gmail.com> parents: 51770 diff changeset	1017 def get_data(self, path):
ee7029710c05 Refactor source and bytecode file loaders in importlib so that there Brett Cannon <bcannon@gmail.com> parents: 51770 diff changeset	1018 """Return the data from path as raw bytes."""
64407 f8ba9558adbf Remove redundant context manager. Florent Xicluna <florent.xicluna@gmail.com> parents: 64291 diff changeset	1019 with _io.FileIO(path, 'r') as file:
62756 34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	1020 return file.read()
51771 ee7029710c05 Refactor source and bytecode file loaders in importlib so that there Brett Cannon <bcannon@gmail.com> parents: 51770 diff changeset	1021
ee7029710c05 Refactor source and bytecode file loaders in importlib so that there Brett Cannon <bcannon@gmail.com> parents: 51770 diff changeset	1022
76478 1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and Brett Cannon <brett@python.org> parents: 76472 diff changeset	1023 class SourceFileLoader(FileLoader, SourceLoader):
51771 ee7029710c05 Refactor source and bytecode file loaders in importlib so that there Brett Cannon <bcannon@gmail.com> parents: 51770 diff changeset	1024
62756 34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	1025 """Concrete implementation of SourceLoader using the file system."""
51771 ee7029710c05 Refactor source and bytecode file loaders in importlib so that there Brett Cannon <bcannon@gmail.com> parents: 51770 diff changeset	1026
74358 87331661042b Issue #13645: pyc files now contain the size of the corresponding source Antoine Pitrou <solipsis@pitrou.net> parents: 73977 diff changeset	1027 def path_stats(self, path):
87331661042b Issue #13645: pyc files now contain the size of the corresponding source Antoine Pitrou <solipsis@pitrou.net> parents: 73977 diff changeset	1028 """Return the metadat for the path."""
87331661042b Issue #13645: pyc files now contain the size of the corresponding source Antoine Pitrou <solipsis@pitrou.net> parents: 73977 diff changeset	1029 st = _os.stat(path)
87331661042b Issue #13645: pyc files now contain the size of the corresponding source Antoine Pitrou <solipsis@pitrou.net> parents: 73977 diff changeset	1030 return {'mtime': st.st_mtime, 'size': st.st_size}
51771 ee7029710c05 Refactor source and bytecode file loaders in importlib so that there Brett Cannon <bcannon@gmail.com> parents: 51770 diff changeset	1031
62756 34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	1032 def set_data(self, path, data):
62587 b07b2c954a80 Move importlib.abc.SourceLoader to _bootstrap. Brett Cannon <bcannon@gmail.com> parents: 61692 diff changeset	1033 """Write bytes data to a file."""
76449 163360370ad2 Make path manipulation more robust for platforms with alternative path Brett Cannon <brett@python.org> parents: 76445 diff changeset	1034 parent, filename = _path_split(path)
64240 cf399de12b54 One of the joys of having test_multiprocessing occasionally execute after Brett Cannon <bcannon@gmail.com> parents: 62761 diff changeset	1035 path_parts = []
cf399de12b54 One of the joys of having test_multiprocessing occasionally execute after Brett Cannon <bcannon@gmail.com> parents: 62761 diff changeset	1036 # Figure out what directories are missing.
cf399de12b54 One of the joys of having test_multiprocessing occasionally execute after Brett Cannon <bcannon@gmail.com> parents: 62761 diff changeset	1037 while parent and not _path_isdir(parent):
76449 163360370ad2 Make path manipulation more robust for platforms with alternative path Brett Cannon <brett@python.org> parents: 76445 diff changeset	1038 parent, part = _path_split(parent)
64240 cf399de12b54 One of the joys of having test_multiprocessing occasionally execute after Brett Cannon <bcannon@gmail.com> parents: 62761 diff changeset	1039 path_parts.append(part)
cf399de12b54 One of the joys of having test_multiprocessing occasionally execute after Brett Cannon <bcannon@gmail.com> parents: 62761 diff changeset	1040 # Create needed directories.
cf399de12b54 One of the joys of having test_multiprocessing occasionally execute after Brett Cannon <bcannon@gmail.com> parents: 62761 diff changeset	1041 for part in reversed(path_parts):
cf399de12b54 One of the joys of having test_multiprocessing occasionally execute after Brett Cannon <bcannon@gmail.com> parents: 62761 diff changeset	1042 parent = _path_join(parent, part)
cf399de12b54 One of the joys of having test_multiprocessing occasionally execute after Brett Cannon <bcannon@gmail.com> parents: 62761 diff changeset	1043 try:
cf399de12b54 One of the joys of having test_multiprocessing occasionally execute after Brett Cannon <bcannon@gmail.com> parents: 62761 diff changeset	1044 _os.mkdir(parent)
73169 e4d44c2e8e81 Simplify and remove few dependencies on 'errno', thanks to PEP 3151. Florent Xicluna <florent.xicluna@gmail.com> parents: 73016 diff changeset	1045 except FileExistsError:
64240 cf399de12b54 One of the joys of having test_multiprocessing occasionally execute after Brett Cannon <bcannon@gmail.com> parents: 62761 diff changeset	1046 # Probably another Python process already created the dir.
73169 e4d44c2e8e81 Simplify and remove few dependencies on 'errno', thanks to PEP 3151. Florent Xicluna <florent.xicluna@gmail.com> parents: 73016 diff changeset	1047 continue
e4d44c2e8e81 Simplify and remove few dependencies on 'errno', thanks to PEP 3151. Florent Xicluna <florent.xicluna@gmail.com> parents: 73016 diff changeset	1048 except PermissionError:
64240 cf399de12b54 One of the joys of having test_multiprocessing occasionally execute after Brett Cannon <bcannon@gmail.com> parents: 62761 diff changeset	1049 # If can't get proper access, then just forget about writing
cf399de12b54 One of the joys of having test_multiprocessing occasionally execute after Brett Cannon <bcannon@gmail.com> parents: 62761 diff changeset	1050 # the data.
73169 e4d44c2e8e81 Simplify and remove few dependencies on 'errno', thanks to PEP 3151. Florent Xicluna <florent.xicluna@gmail.com> parents: 73016 diff changeset	1051 return
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	1052 try:
72960 c16063765d3a Issue #13146: Writing a pyc file is now atomic under POSIX. Antoine Pitrou <solipsis@pitrou.net> parents: 68887 diff changeset	1053 _write_atomic(path, data)
76753 59b8a7fcb047 Properly mark names in importlib._bootstrap as private. Brett Cannon <brett@python.org> parents: 76586 diff changeset	1054 _verbose_message('created {!r}', path)
75185 27d31f0c4ad5 Issue #14077: importlib: Fix regression introduced by de6703671386. Charles-François Natali <neologix@free.fr> parents: 75082 diff changeset	1055 except (PermissionError, FileExistsError):
27d31f0c4ad5 Issue #14077: importlib: Fix regression introduced by de6703671386. Charles-François Natali <neologix@free.fr> parents: 75082 diff changeset	1056 # Don't worry if you can't write bytecode or someone is writing
27d31f0c4ad5 Issue #14077: importlib: Fix regression introduced by de6703671386. Charles-François Natali <neologix@free.fr> parents: 75082 diff changeset	1057 # it at the same time.
73016 170ed6735d4b Silence the FileExistsError which can be raised because of the O_EXCL flag Antoine Pitrou <solipsis@pitrou.net> parents: 72960 diff changeset	1058 pass
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	1059
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	1060
76548 a2cf07135e4f Issue #14605: Rename _SourcelessFileLoader to SourcelessFileLoader. Marc-Andre Lemburg <mal@egenix.com> parents: 76547 diff changeset	1061 class SourcelessFileLoader(FileLoader, _LoaderBasics):
55355 dfaf6de2c8ce Implement the PEP 302 protocol for get_filename() as Brett Cannon <bcannon@gmail.com> parents: 52633 diff changeset	1062
62756 34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	1063 """Loader which handles sourceless file imports."""
52112 03080fa08dfe Implement get_source for importlib.abc.PyLoader using source_path and get_data. Brett Cannon <bcannon@gmail.com> parents: 52083 diff changeset	1064
62756 34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	1065 def load_module(self, fullname):
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	1066 return self._load_module(fullname, sourceless=True)
51771 ee7029710c05 Refactor source and bytecode file loaders in importlib so that there Brett Cannon <bcannon@gmail.com> parents: 51770 diff changeset	1067
ee7029710c05 Refactor source and bytecode file loaders in importlib so that there Brett Cannon <bcannon@gmail.com> parents: 51770 diff changeset	1068 def get_code(self, fullname):
62756 34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	1069 path = self.get_filename(fullname)
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	1070 data = self.get_data(path)
76280 c071af95772a Have importlib take advantage of ImportError's new 'name' and 'path' Brett Cannon <brett@python.org> parents: 76081 diff changeset	1071 bytes_data = self._bytes_from_bytecode(fullname, data, path, None)
62756 34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	1072 found = marshal.loads(bytes_data)
76753 59b8a7fcb047 Properly mark names in importlib._bootstrap as private. Brett Cannon <brett@python.org> parents: 76586 diff changeset	1073 if isinstance(found, _code_type):
59b8a7fcb047 Properly mark names in importlib._bootstrap as private. Brett Cannon <brett@python.org> parents: 76586 diff changeset	1074 _verbose_message('code object from {!r}', path)
62756 34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	1075 return found
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	1076 else:
76280 c071af95772a Have importlib take advantage of ImportError's new 'name' and 'path' Brett Cannon <brett@python.org> parents: 76081 diff changeset	1077 raise ImportError("Non-code object in {}".format(path),
c071af95772a Have importlib take advantage of ImportError's new 'name' and 'path' Brett Cannon <brett@python.org> parents: 76081 diff changeset	1078 name=fullname, path=path)
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	1079
62756 34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	1080 def get_source(self, fullname):
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	1081 """Return None as there is no source code."""
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	1082 return None
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	1083
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	1084
78486 1db6553f3f8c Issue #15576: Allow extension modules to be a package's __init__ Brett Cannon <brett@python.org> parents: 78485 diff changeset	1085 # Filled in by _setup().
1db6553f3f8c Issue #15576: Allow extension modules to be a package's __init__ Brett Cannon <brett@python.org> parents: 78485 diff changeset	1086 EXTENSION_SUFFIXES = []
1db6553f3f8c Issue #15576: Allow extension modules to be a package's __init__ Brett Cannon <brett@python.org> parents: 78485 diff changeset	1087
1db6553f3f8c Issue #15576: Allow extension modules to be a package's __init__ Brett Cannon <brett@python.org> parents: 78485 diff changeset	1088
76478 1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and Brett Cannon <brett@python.org> parents: 76472 diff changeset	1089 class ExtensionFileLoader:
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	1090
52141 f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1091 """Loader for extension modules.
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	1092
52141 f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1093 The constructor is designed to work with FileFinder.
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	1094
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	1095 """
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	1096
62756 34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	1097 def __init__(self, name, path):
76478 1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and Brett Cannon <brett@python.org> parents: 76472 diff changeset	1098 self.name = name
1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and Brett Cannon <brett@python.org> parents: 76472 diff changeset	1099 self.path = path
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	1100
52141 f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1101 @_check_name
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1102 @set_package
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1103 @set_loader
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1104 def load_module(self, fullname):
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1105 """Load an extension module."""
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1106 is_reload = fullname in sys.modules
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1107 try:
78344 62033490ca0f Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise Nick Coghlan <ncoghlan@gmail.com> parents: 78314 diff changeset	1108 module = _call_with_frames_removed(_imp.load_dynamic,
62033490ca0f Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise Nick Coghlan <ncoghlan@gmail.com> parents: 78314 diff changeset	1109 fullname, self.path)
76753 59b8a7fcb047 Properly mark names in importlib._bootstrap as private. Brett Cannon <brett@python.org> parents: 76586 diff changeset	1110 _verbose_message('extension module loaded from {!r}', self.path)
78486 1db6553f3f8c Issue #15576: Allow extension modules to be a package's __init__ Brett Cannon <brett@python.org> parents: 78485 diff changeset	1111 if self.is_package(fullname):
1db6553f3f8c Issue #15576: Allow extension modules to be a package's __init__ Brett Cannon <brett@python.org> parents: 78485 diff changeset	1112 module.__path__ = [_path_split(self.path)[0]]
76303 2dd046be2c88 Issue #2377: Make importlib the implementation of __import__(). Brett Cannon <brett@python.org> parents: 76280 diff changeset	1113 return module
52141 f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1114 except:
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1115 if not is_reload and fullname in sys.modules:
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1116 del sys.modules[fullname]
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1117 raise
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1118
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1119 def is_package(self, fullname):
78487 e024f6ba5ed8 update docstring per the extension package fix, refactor Philip Jenvey <pjenvey@underboss.org> parents: 78486 diff changeset	1120 """Return if the extension module is a package."""
78486 1db6553f3f8c Issue #15576: Allow extension modules to be a package's __init__ Brett Cannon <brett@python.org> parents: 78485 diff changeset	1121 file_name = _path_split(self.path)[1]
78487 e024f6ba5ed8 update docstring per the extension package fix, refactor Philip Jenvey <pjenvey@underboss.org> parents: 78486 diff changeset	1122 return any(file_name == '__init__' + suffix
e024f6ba5ed8 update docstring per the extension package fix, refactor Philip Jenvey <pjenvey@underboss.org> parents: 78486 diff changeset	1123 for suffix in EXTENSION_SUFFIXES)
52141 f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1124
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1125 def get_code(self, fullname):
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1126 """Return None as an extension module cannot create a code object."""
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1127 return None
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1128
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1129 def get_source(self, fullname):
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1130 """Return None as extension modules have no source code."""
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1131 return None
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	1132
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	1133
77126 702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1134 class _NamespacePath:
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1135 """Represents a namespace package's path. It uses the module name
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1136 to find its parent module, and from there it looks up the parent's
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1137 __path__. When this changes, the module's own path is recomputed,
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1138 using path_finder. For top-leve modules, the parent module's path
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1139 is sys.path."""
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1140
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1141 def __init__(self, name, path, path_finder):
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1142 self._name = name
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1143 self._path = path
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1144 self._last_parent_path = tuple(self._get_parent_path())
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1145 self._path_finder = path_finder
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1146
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1147 def _find_parent_path_names(self):
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1148 """Returns a tuple of (parent-module-name, parent-path-attr-name)"""
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1149 parent, dot, me = self._name.rpartition('.')
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1150 if dot == '':
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1151 # This is a top-level module. sys.path contains the parent path.
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1152 return 'sys', 'path'
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1153 # Not a top-level module. parent-module.__path__ contains the
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1154 # parent path.
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1155 return parent, '__path__'
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1156
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1157 def _get_parent_path(self):
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1158 parent_module_name, path_attr_name = self._find_parent_path_names()
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1159 return getattr(sys.modules[parent_module_name], path_attr_name)
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1160
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1161 def _recalculate(self):
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1162 # If the parent's path has changed, recalculate _path
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1163 parent_path = tuple(self._get_parent_path()) # Make a copy
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1164 if parent_path != self._last_parent_path:
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1165 loader, new_path = self._path_finder(self._name, parent_path)
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1166 # Note that no changes are made if a loader is returned, but we
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1167 # do remember the new parent path
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1168 if loader is None:
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1169 self._path = new_path
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1170 self._last_parent_path = parent_path # Save the copy
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1171 return self._path
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1172
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1173 def __iter__(self):
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1174 return iter(self._recalculate())
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1175
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1176 def __len__(self):
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1177 return len(self._recalculate())
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1178
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1179 def __repr__(self):
77964 37850f083af5 Minor refactoring in importlib._bootstrap, and fix the '_wrap' docstring. Florent Xicluna <florent.xicluna@gmail.com> parents: 77917 diff changeset	1180 return "_NamespacePath({!r})".format(self._path)
77126 702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1181
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1182 def __contains__(self, item):
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1183 return item in self._recalculate()
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1184
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1185 def append(self, item):
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1186 self._path.append(item)
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1187
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1188
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1189 class NamespaceLoader:
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1190 def __init__(self, name, path, path_finder):
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1191 self._path = _NamespacePath(name, path, path_finder)
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1192
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1193 @classmethod
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1194 def module_repr(cls, module):
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1195 return "<module '{}' (namespace)>".format(module.__name__)
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1196
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1197 @module_for_loader
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1198 def load_module(self, module):
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1199 """Load a namespace module."""
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1200 _verbose_message('namespace module loaded with path {!r}', self._path)
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1201 module.__path__ = self._path
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1202 return module
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1203
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1204
52141 f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1205 # Finders #####################################################################
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	1206
51322 53d6445c6c5a Rename importlib._bootstrap.SysPathFinder to PathFinder and expose off of Brett Cannon <bcannon@gmail.com> parents: 51266 diff changeset	1207 class PathFinder:
51266 63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to Brett Cannon <bcannon@gmail.com> parents: 51242 diff changeset	1208
78382 1f8351cf00f3 Issue #15502: Bring the importlib.PathFinder docs and docstring more in line with the new import system documentation, and fix various parts of the new docs that weren't quite right given PEP 420 or were otherwise a bit misleading. Also note the key terminology problem still being discussed in the issue Nick Coghlan <ncoghlan@gmail.com> parents: 78380 diff changeset	1209 """Meta path finder for sys.path and package __path__ attributes."""
51266 63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to Brett Cannon <bcannon@gmail.com> parents: 51242 diff changeset	1210
51639 ef2cba80c975 Change importlib.machinery.PathFinder to not have implicit semantics (that's Brett Cannon <bcannon@gmail.com> parents: 51433 diff changeset	1211 @classmethod
78485 0a75ce232f56 Issue #15502: Finish bringing importlib.abc in line with the current Brett Cannon <brett@python.org> parents: 78456 diff changeset	1212 def invalidate_caches(cls):
0a75ce232f56 Issue #15502: Finish bringing importlib.abc in line with the current Brett Cannon <brett@python.org> parents: 78456 diff changeset	1213 """Call the invalidate_caches() method on all path entry finders
0a75ce232f56 Issue #15502: Finish bringing importlib.abc in line with the current Brett Cannon <brett@python.org> parents: 78456 diff changeset	1214 stored in sys.path_importer_caches (where implemented)."""
0a75ce232f56 Issue #15502: Finish bringing importlib.abc in line with the current Brett Cannon <brett@python.org> parents: 78456 diff changeset	1215 for finder in sys.path_importer_cache.values():
0a75ce232f56 Issue #15502: Finish bringing importlib.abc in line with the current Brett Cannon <brett@python.org> parents: 78456 diff changeset	1216 if hasattr(finder, 'invalidate_caches'):
0a75ce232f56 Issue #15502: Finish bringing importlib.abc in line with the current Brett Cannon <brett@python.org> parents: 78456 diff changeset	1217 finder.invalidate_caches()
0a75ce232f56 Issue #15502: Finish bringing importlib.abc in line with the current Brett Cannon <brett@python.org> parents: 78456 diff changeset	1218
0a75ce232f56 Issue #15502: Finish bringing importlib.abc in line with the current Brett Cannon <brett@python.org> parents: 78456 diff changeset	1219 @classmethod
76566 57d558f1904d Issue #14605: Make explicit the entries on sys.path_hooks that used to Brett Cannon <brett@python.org> parents: 76549 diff changeset	1220 def _path_hooks(cls, path):
51639 ef2cba80c975 Change importlib.machinery.PathFinder to not have implicit semantics (that's Brett Cannon <bcannon@gmail.com> parents: 51433 diff changeset	1221 """Search sequence of hooks for a finder for 'path'.
51266 63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to Brett Cannon <bcannon@gmail.com> parents: 51242 diff changeset	1222
51639 ef2cba80c975 Change importlib.machinery.PathFinder to not have implicit semantics (that's Brett Cannon <bcannon@gmail.com> parents: 51433 diff changeset	1223 If 'hooks' is false then use sys.path_hooks.
51266 63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to Brett Cannon <bcannon@gmail.com> parents: 51242 diff changeset	1224
63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to Brett Cannon <bcannon@gmail.com> parents: 51242 diff changeset	1225 """
76566 57d558f1904d Issue #14605: Make explicit the entries on sys.path_hooks that used to Brett Cannon <brett@python.org> parents: 76549 diff changeset	1226 if not sys.path_hooks:
57d558f1904d Issue #14605: Make explicit the entries on sys.path_hooks that used to Brett Cannon <brett@python.org> parents: 76549 diff changeset	1227 _warnings.warn('sys.path_hooks is empty', ImportWarning)
57d558f1904d Issue #14605: Make explicit the entries on sys.path_hooks that used to Brett Cannon <brett@python.org> parents: 76549 diff changeset	1228 for hook in sys.path_hooks:
51266 63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to Brett Cannon <bcannon@gmail.com> parents: 51242 diff changeset	1229 try:
63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to Brett Cannon <bcannon@gmail.com> parents: 51242 diff changeset	1230 return hook(path)
63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to Brett Cannon <bcannon@gmail.com> parents: 51242 diff changeset	1231 except ImportError:
63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to Brett Cannon <bcannon@gmail.com> parents: 51242 diff changeset	1232 continue
63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to Brett Cannon <bcannon@gmail.com> parents: 51242 diff changeset	1233 else:
76582 7025ee00dbf6 Issue #14605: Use None in sys.path_importer_cache to represent no Brett Cannon <brett@python.org> parents: 76579 diff changeset	1234 return None
51266 63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to Brett Cannon <bcannon@gmail.com> parents: 51242 diff changeset	1235
51322 53d6445c6c5a Rename importlib._bootstrap.SysPathFinder to PathFinder and expose off of Brett Cannon <bcannon@gmail.com> parents: 51266 diff changeset	1236 @classmethod
76566 57d558f1904d Issue #14605: Make explicit the entries on sys.path_hooks that used to Brett Cannon <brett@python.org> parents: 76549 diff changeset	1237 def _path_importer_cache(cls, path):
78382 1f8351cf00f3 Issue #15502: Bring the importlib.PathFinder docs and docstring more in line with the new import system documentation, and fix various parts of the new docs that weren't quite right given PEP 420 or were otherwise a bit misleading. Also note the key terminology problem still being discussed in the issue Nick Coghlan <ncoghlan@gmail.com> parents: 78380 diff changeset	1238 """Get the finder for the path entry from sys.path_importer_cache.
51266 63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to Brett Cannon <bcannon@gmail.com> parents: 51242 diff changeset	1239
78382 1f8351cf00f3 Issue #15502: Bring the importlib.PathFinder docs and docstring more in line with the new import system documentation, and fix various parts of the new docs that weren't quite right given PEP 420 or were otherwise a bit misleading. Also note the key terminology problem still being discussed in the issue Nick Coghlan <ncoghlan@gmail.com> parents: 78380 diff changeset	1240 If the path entry is not in the cache, find the appropriate finder
1f8351cf00f3 Issue #15502: Bring the importlib.PathFinder docs and docstring more in line with the new import system documentation, and fix various parts of the new docs that weren't quite right given PEP 420 or were otherwise a bit misleading. Also note the key terminology problem still being discussed in the issue Nick Coghlan <ncoghlan@gmail.com> parents: 78380 diff changeset	1241 and cache it. If no finder is available, store None.
51266 63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to Brett Cannon <bcannon@gmail.com> parents: 51242 diff changeset	1242
63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to Brett Cannon <bcannon@gmail.com> parents: 51242 diff changeset	1243 """
74994 b8593ec7e8c5 Tweak the handling of the empty string in sys.path for importlib. Brett Cannon <brett@python.org> parents: 74992 diff changeset	1244 if path == '':
75048 bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function. Antoine Pitrou <solipsis@pitrou.net> parents: 75045 diff changeset	1245 path = '.'
51266 63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to Brett Cannon <bcannon@gmail.com> parents: 51242 diff changeset	1246 try:
51322 53d6445c6c5a Rename importlib._bootstrap.SysPathFinder to PathFinder and expose off of Brett Cannon <bcannon@gmail.com> parents: 51266 diff changeset	1247 finder = sys.path_importer_cache[path]
51266 63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to Brett Cannon <bcannon@gmail.com> parents: 51242 diff changeset	1248 except KeyError:
74994 b8593ec7e8c5 Tweak the handling of the empty string in sys.path for importlib. Brett Cannon <brett@python.org> parents: 74992 diff changeset	1249 finder = cls._path_hooks(path)
51266 63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to Brett Cannon <bcannon@gmail.com> parents: 51242 diff changeset	1250 sys.path_importer_cache[path] = finder
63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to Brett Cannon <bcannon@gmail.com> parents: 51242 diff changeset	1251 return finder
63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to Brett Cannon <bcannon@gmail.com> parents: 51242 diff changeset	1252
51322 53d6445c6c5a Rename importlib._bootstrap.SysPathFinder to PathFinder and expose off of Brett Cannon <bcannon@gmail.com> parents: 51266 diff changeset	1253 @classmethod
77126 702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1254 def _get_loader(cls, fullname, path):
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1255 """Find the loader or namespace_path for this module/package name."""
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1256 # If this ends up being a namespace package, namespace_path is
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1257 # the list of paths that will become its __path__
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1258 namespace_path = []
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1259 for entry in path:
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1260 finder = cls._path_importer_cache(entry)
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1261 if finder is not None:
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1262 if hasattr(finder, 'find_loader'):
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1263 loader, portions = finder.find_loader(fullname)
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1264 else:
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1265 loader = finder.find_module(fullname)
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1266 portions = []
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1267 if loader is not None:
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1268 # We found a loader: return it immediately.
78485 0a75ce232f56 Issue #15502: Finish bringing importlib.abc in line with the current Brett Cannon <brett@python.org> parents: 78456 diff changeset	1269 return loader, namespace_path
77126 702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1270 # This is possibly part of a namespace package.
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1271 # Remember these path entries (if any) for when we
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1272 # create a namespace package, and continue iterating
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1273 # on path.
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1274 namespace_path.extend(portions)
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1275 else:
78485 0a75ce232f56 Issue #15502: Finish bringing importlib.abc in line with the current Brett Cannon <brett@python.org> parents: 78456 diff changeset	1276 return None, namespace_path
77126 702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1277
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1278 @classmethod
51322 53d6445c6c5a Rename importlib._bootstrap.SysPathFinder to PathFinder and expose off of Brett Cannon <bcannon@gmail.com> parents: 51266 diff changeset	1279 def find_module(cls, fullname, path=None):
52165 62f9dad628de A few more docstring/API cleanups for importlib. Brett Cannon <bcannon@gmail.com> parents: 52164 diff changeset	1280 """Find the module on sys.path or 'path' based on sys.path_hooks and
62f9dad628de A few more docstring/API cleanups for importlib. Brett Cannon <bcannon@gmail.com> parents: 52164 diff changeset	1281 sys.path_importer_cache."""
76384 c1399cf7bd6a Issue #12599: Be more strict in accepting None vs. a false-like object Brett Cannon <brett@python.org> parents: 76331 diff changeset	1282 if path is None:
51266 63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to Brett Cannon <bcannon@gmail.com> parents: 51242 diff changeset	1283 path = sys.path
77126 702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1284 loader, namespace_path = cls._get_loader(fullname, path)
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1285 if loader is not None:
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1286 return loader
51266 63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to Brett Cannon <bcannon@gmail.com> parents: 51242 diff changeset	1287 else:
77126 702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1288 if namespace_path:
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1289 # We found at least one namespace path. Return a
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1290 # loader which can create the namespace package.
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1291 return NamespaceLoader(fullname, namespace_path, cls._get_loader)
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1292 else:
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1293 return None
51266 63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to Brett Cannon <bcannon@gmail.com> parents: 51242 diff changeset	1294
63386ae77f79 To prevent another screw-up on my part where my prototype gets lost thanks to Brett Cannon <bcannon@gmail.com> parents: 51242 diff changeset	1295
76478 1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and Brett Cannon <brett@python.org> parents: 76472 diff changeset	1296 class FileFinder:
52141 f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1297
62756 34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	1298 """File-based finder.
52141 f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1299
76478 1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and Brett Cannon <brett@python.org> parents: 76472 diff changeset	1300 Interactions with the file system are cached for performance, being
1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and Brett Cannon <brett@python.org> parents: 76472 diff changeset	1301 refreshed when the directory the finder is handling has been modified.
52141 f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1302
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1303 """
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1304
62756 34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	1305 def __init__(self, path, *details):
76478 1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and Brett Cannon <brett@python.org> parents: 76472 diff changeset	1306 """Initialize with the path to search on and a variable number of
77126 702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1307 3-tuples containing the loader, file suffixes the loader recognizes,
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1308 and a boolean of whether the loader handles packages."""
78486 1db6553f3f8c Issue #15576: Allow extension modules to be a package's __init__ Brett Cannon <brett@python.org> parents: 78485 diff changeset	1309 loaders = []
1db6553f3f8c Issue #15576: Allow extension modules to be a package's __init__ Brett Cannon <brett@python.org> parents: 78485 diff changeset	1310 for loader, suffixes in details:
1db6553f3f8c Issue #15576: Allow extension modules to be a package's __init__ Brett Cannon <brett@python.org> parents: 78485 diff changeset	1311 loaders.extend((suffix, loader) for suffix in suffixes)
1db6553f3f8c Issue #15576: Allow extension modules to be a package's __init__ Brett Cannon <brett@python.org> parents: 78485 diff changeset	1312 self._loaders = loaders
75048 bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function. Antoine Pitrou <solipsis@pitrou.net> parents: 75045 diff changeset	1313 # Base (directory) path
bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function. Antoine Pitrou <solipsis@pitrou.net> parents: 75045 diff changeset	1314 self.path = path or '.'
bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function. Antoine Pitrou <solipsis@pitrou.net> parents: 75045 diff changeset	1315 self._path_mtime = -1
bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function. Antoine Pitrou <solipsis@pitrou.net> parents: 75045 diff changeset	1316 self._path_cache = set()
75082 3297dcdad196 Issue #14063: fix test_importlib failure under OS X case-insensitive filesystems Antoine Pitrou <solipsis@pitrou.net> parents: 75070 diff changeset	1317 self._relaxed_path_cache = set()
75324 de125ee32897 Update importlib.invalidate_caches() to be more general. Brett Cannon <brett@python.org> parents: 75250 diff changeset	1318
de125ee32897 Update importlib.invalidate_caches() to be more general. Brett Cannon <brett@python.org> parents: 75250 diff changeset	1319 def invalidate_caches(self):
de125ee32897 Update importlib.invalidate_caches() to be more general. Brett Cannon <brett@python.org> parents: 75250 diff changeset	1320 """Invalidate the directory mtime."""
de125ee32897 Update importlib.invalidate_caches() to be more general. Brett Cannon <brett@python.org> parents: 75250 diff changeset	1321 self._path_mtime = -1
52141 f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1322
78489 e7a67f1bf604 Issue #15502: Refactor some code. Brett Cannon <brett@python.org> parents: 78487 diff changeset	1323 find_module = _find_module_shim
77126 702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1324
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1325 def find_loader(self, fullname):
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1326 """Try to find a loader for the specified module, or the namespace
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1327 package portions. Returns (loader, list-of-portions)."""
77757 2051fead1933 Fixes issue 15039: namespace packages are no longer imported in preference to modules of the same name. Eric V. Smith <eric@trueblade.com> parents: 77600 diff changeset	1328 is_namespace = False
52141 f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1329 tail_module = fullname.rpartition('.')[2]
75048 bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function. Antoine Pitrou <solipsis@pitrou.net> parents: 75045 diff changeset	1330 try:
bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function. Antoine Pitrou <solipsis@pitrou.net> parents: 75045 diff changeset	1331 mtime = _os.stat(self.path).st_mtime
bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function. Antoine Pitrou <solipsis@pitrou.net> parents: 75045 diff changeset	1332 except OSError:
bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function. Antoine Pitrou <solipsis@pitrou.net> parents: 75045 diff changeset	1333 mtime = -1
75324 de125ee32897 Update importlib.invalidate_caches() to be more general. Brett Cannon <brett@python.org> parents: 75250 diff changeset	1334 if mtime != self._path_mtime:
75048 bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function. Antoine Pitrou <solipsis@pitrou.net> parents: 75045 diff changeset	1335 self._fill_cache()
bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function. Antoine Pitrou <solipsis@pitrou.net> parents: 75045 diff changeset	1336 self._path_mtime = mtime
75082 3297dcdad196 Issue #14063: fix test_importlib failure under OS X case-insensitive filesystems Antoine Pitrou <solipsis@pitrou.net> parents: 75070 diff changeset	1337 # tail_module keeps the original casing, for __file__ and friends
3297dcdad196 Issue #14063: fix test_importlib failure under OS X case-insensitive filesystems Antoine Pitrou <solipsis@pitrou.net> parents: 75070 diff changeset	1338 if _relax_case():
3297dcdad196 Issue #14063: fix test_importlib failure under OS X case-insensitive filesystems Antoine Pitrou <solipsis@pitrou.net> parents: 75070 diff changeset	1339 cache = self._relaxed_path_cache
3297dcdad196 Issue #14063: fix test_importlib failure under OS X case-insensitive filesystems Antoine Pitrou <solipsis@pitrou.net> parents: 75070 diff changeset	1340 cache_module = tail_module.lower()
3297dcdad196 Issue #14063: fix test_importlib failure under OS X case-insensitive filesystems Antoine Pitrou <solipsis@pitrou.net> parents: 75070 diff changeset	1341 else:
3297dcdad196 Issue #14063: fix test_importlib failure under OS X case-insensitive filesystems Antoine Pitrou <solipsis@pitrou.net> parents: 75070 diff changeset	1342 cache = self._path_cache
3297dcdad196 Issue #14063: fix test_importlib failure under OS X case-insensitive filesystems Antoine Pitrou <solipsis@pitrou.net> parents: 75070 diff changeset	1343 cache_module = tail_module
76309 e2f48b076012 Add some comments. Brett Cannon <brett@python.org> parents: 76303 diff changeset	1344 # Check if the module is the name of a directory (and thus a package).
75082 3297dcdad196 Issue #14063: fix test_importlib failure under OS X case-insensitive filesystems Antoine Pitrou <solipsis@pitrou.net> parents: 75070 diff changeset	1345 if cache_module in cache:
75048 bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function. Antoine Pitrou <solipsis@pitrou.net> parents: 75045 diff changeset	1346 base_path = _path_join(self.path, tail_module)
bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function. Antoine Pitrou <solipsis@pitrou.net> parents: 75045 diff changeset	1347 if _path_isdir(base_path):
78486 1db6553f3f8c Issue #15576: Allow extension modules to be a package's __init__ Brett Cannon <brett@python.org> parents: 78485 diff changeset	1348 for suffix, loader in self._loaders:
75048 bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function. Antoine Pitrou <solipsis@pitrou.net> parents: 75045 diff changeset	1349 init_filename = '__init__' + suffix
bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function. Antoine Pitrou <solipsis@pitrou.net> parents: 75045 diff changeset	1350 full_path = _path_join(base_path, init_filename)
bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function. Antoine Pitrou <solipsis@pitrou.net> parents: 75045 diff changeset	1351 if _path_isfile(full_path):
77126 702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1352 return (loader(fullname, full_path), [base_path])
75048 bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function. Antoine Pitrou <solipsis@pitrou.net> parents: 75045 diff changeset	1353 else:
77757 2051fead1933 Fixes issue 15039: namespace packages are no longer imported in preference to modules of the same name. Eric V. Smith <eric@trueblade.com> parents: 77600 diff changeset	1354 # A namespace package, return the path if we don't also
2051fead1933 Fixes issue 15039: namespace packages are no longer imported in preference to modules of the same name. Eric V. Smith <eric@trueblade.com> parents: 77600 diff changeset	1355 # find a module in the next section.
2051fead1933 Fixes issue 15039: namespace packages are no longer imported in preference to modules of the same name. Eric V. Smith <eric@trueblade.com> parents: 77600 diff changeset	1356 is_namespace = True
76309 e2f48b076012 Add some comments. Brett Cannon <brett@python.org> parents: 76303 diff changeset	1357 # Check for a file w/ a proper suffix exists.
78486 1db6553f3f8c Issue #15576: Allow extension modules to be a package's __init__ Brett Cannon <brett@python.org> parents: 78485 diff changeset	1358 for suffix, loader in self._loaders:
75082 3297dcdad196 Issue #14063: fix test_importlib failure under OS X case-insensitive filesystems Antoine Pitrou <solipsis@pitrou.net> parents: 75070 diff changeset	1359 if cache_module + suffix in cache:
3297dcdad196 Issue #14063: fix test_importlib failure under OS X case-insensitive filesystems Antoine Pitrou <solipsis@pitrou.net> parents: 75070 diff changeset	1360 full_path = _path_join(self.path, tail_module + suffix)
75048 bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function. Antoine Pitrou <solipsis@pitrou.net> parents: 75045 diff changeset	1361 if _path_isfile(full_path):
77126 702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1362 return (loader(fullname, full_path), [])
77757 2051fead1933 Fixes issue 15039: namespace packages are no longer imported in preference to modules of the same name. Eric V. Smith <eric@trueblade.com> parents: 77600 diff changeset	1363 if is_namespace:
2051fead1933 Fixes issue 15039: namespace packages are no longer imported in preference to modules of the same name. Eric V. Smith <eric@trueblade.com> parents: 77600 diff changeset	1364 return (None, [base_path])
77126 702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 77013 diff changeset	1365 return (None, [])
62756 34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	1366
75048 bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function. Antoine Pitrou <solipsis@pitrou.net> parents: 75045 diff changeset	1367 def _fill_cache(self):
bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function. Antoine Pitrou <solipsis@pitrou.net> parents: 75045 diff changeset	1368 """Fill the cache of potential modules and packages for this directory."""
bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function. Antoine Pitrou <solipsis@pitrou.net> parents: 75045 diff changeset	1369 path = self.path
bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function. Antoine Pitrou <solipsis@pitrou.net> parents: 75045 diff changeset	1370 contents = _os.listdir(path)
75082 3297dcdad196 Issue #14063: fix test_importlib failure under OS X case-insensitive filesystems Antoine Pitrou <solipsis@pitrou.net> parents: 75070 diff changeset	1371 # We store two cached versions, to handle runtime changes of the
3297dcdad196 Issue #14063: fix test_importlib failure under OS X case-insensitive filesystems Antoine Pitrou <solipsis@pitrou.net> parents: 75070 diff changeset	1372 # PYTHONCASEOK environment variable.
76428 a32be109bd86 Issue #14581: Windows users are allowed to import modules w/o taking Brett Cannon <brett@python.org> parents: 76406 diff changeset	1373 if not sys.platform.startswith('win'):
a32be109bd86 Issue #14581: Windows users are allowed to import modules w/o taking Brett Cannon <brett@python.org> parents: 76406 diff changeset	1374 self._path_cache = set(contents)
a32be109bd86 Issue #14581: Windows users are allowed to import modules w/o taking Brett Cannon <brett@python.org> parents: 76406 diff changeset	1375 else:
a32be109bd86 Issue #14581: Windows users are allowed to import modules w/o taking Brett Cannon <brett@python.org> parents: 76406 diff changeset	1376 # Windows users can import modules with case-insensitive file
a32be109bd86 Issue #14581: Windows users are allowed to import modules w/o taking Brett Cannon <brett@python.org> parents: 76406 diff changeset	1377 # suffixes (for legacy reasons). Make the suffix lowercase here
a32be109bd86 Issue #14581: Windows users are allowed to import modules w/o taking Brett Cannon <brett@python.org> parents: 76406 diff changeset	1378 # so it's done once instead of for every import. This is safe as
a32be109bd86 Issue #14581: Windows users are allowed to import modules w/o taking Brett Cannon <brett@python.org> parents: 76406 diff changeset	1379 # the specified suffixes to check against are always specified in a
a32be109bd86 Issue #14581: Windows users are allowed to import modules w/o taking Brett Cannon <brett@python.org> parents: 76406 diff changeset	1380 # case-sensitive manner.
a32be109bd86 Issue #14581: Windows users are allowed to import modules w/o taking Brett Cannon <brett@python.org> parents: 76406 diff changeset	1381 lower_suffix_contents = set()
a32be109bd86 Issue #14581: Windows users are allowed to import modules w/o taking Brett Cannon <brett@python.org> parents: 76406 diff changeset	1382 for item in contents:
a32be109bd86 Issue #14581: Windows users are allowed to import modules w/o taking Brett Cannon <brett@python.org> parents: 76406 diff changeset	1383 name, dot, suffix = item.partition('.')
a32be109bd86 Issue #14581: Windows users are allowed to import modules w/o taking Brett Cannon <brett@python.org> parents: 76406 diff changeset	1384 if dot:
a32be109bd86 Issue #14581: Windows users are allowed to import modules w/o taking Brett Cannon <brett@python.org> parents: 76406 diff changeset	1385 new_name = '{}.{}'.format(name, suffix.lower())
a32be109bd86 Issue #14581: Windows users are allowed to import modules w/o taking Brett Cannon <brett@python.org> parents: 76406 diff changeset	1386 else:
a32be109bd86 Issue #14581: Windows users are allowed to import modules w/o taking Brett Cannon <brett@python.org> parents: 76406 diff changeset	1387 new_name = name
a32be109bd86 Issue #14581: Windows users are allowed to import modules w/o taking Brett Cannon <brett@python.org> parents: 76406 diff changeset	1388 lower_suffix_contents.add(new_name)
a32be109bd86 Issue #14581: Windows users are allowed to import modules w/o taking Brett Cannon <brett@python.org> parents: 76406 diff changeset	1389 self._path_cache = lower_suffix_contents
76753 59b8a7fcb047 Properly mark names in importlib._bootstrap as private. Brett Cannon <brett@python.org> parents: 76586 diff changeset	1390 if sys.platform.startswith(_CASE_INSENSITIVE_PLATFORMS):
76406 052732bd3c10 don't bother keeping a set we'll never use Benjamin Peterson <benjamin@python.org> parents: 76390 diff changeset	1391 self._relaxed_path_cache = set(fn.lower() for fn in contents)
75048 bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function. Antoine Pitrou <solipsis@pitrou.net> parents: 75045 diff changeset	1392
76478 1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and Brett Cannon <brett@python.org> parents: 76472 diff changeset	1393 @classmethod
1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and Brett Cannon <brett@python.org> parents: 76472 diff changeset	1394 def path_hook(cls, *loader_details):
1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and Brett Cannon <brett@python.org> parents: 76472 diff changeset	1395 """A class method which returns a closure to use on sys.path_hook
1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and Brett Cannon <brett@python.org> parents: 76472 diff changeset	1396 which will return an instance using the specified loaders and the path
1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and Brett Cannon <brett@python.org> parents: 76472 diff changeset	1397 called on the closure.
62756 34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	1398
76478 1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and Brett Cannon <brett@python.org> parents: 76472 diff changeset	1399 If the path called on the closure is not a directory, ImportError is
1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and Brett Cannon <brett@python.org> parents: 76472 diff changeset	1400 raised.
62756 34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib. Brett Cannon <bcannon@gmail.com> parents: 62587 diff changeset	1401
76478 1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and Brett Cannon <brett@python.org> parents: 76472 diff changeset	1402 """
1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and Brett Cannon <brett@python.org> parents: 76472 diff changeset	1403 def path_hook_for_FileFinder(path):
1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and Brett Cannon <brett@python.org> parents: 76472 diff changeset	1404 """Path hook for importlib.machinery.FileFinder."""
1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and Brett Cannon <brett@python.org> parents: 76472 diff changeset	1405 if not _path_isdir(path):
1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and Brett Cannon <brett@python.org> parents: 76472 diff changeset	1406 raise ImportError("only directories are supported", path=path)
1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and Brett Cannon <brett@python.org> parents: 76472 diff changeset	1407 return cls(path, *loader_details)
52141 f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1408
76478 1da623513b26 Issue #14605: Expose importlib.abc.FileLoader and Brett Cannon <brett@python.org> parents: 76472 diff changeset	1409 return path_hook_for_FileFinder
52141 f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1410
77600 df856963fc6c A better repr() for FileFinder Antoine Pitrou <solipsis@pitrou.net> parents: 77457 diff changeset	1411 def __repr__(self):
df856963fc6c A better repr() for FileFinder Antoine Pitrou <solipsis@pitrou.net> parents: 77457 diff changeset	1412 return "FileFinder(%r)" % (self.path,)
52141 f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1413
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1414
52140 1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap. Brett Cannon <bcannon@gmail.com> parents: 52139 diff changeset	1415 # Import itself ###############################################################
1a8bbc5fb3ef Do a little bit of reorganization on importlib._bootstrap. Brett Cannon <bcannon@gmail.com> parents: 52139 diff changeset	1416
52141 f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1417 class _ImportLockContext:
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1418
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1419 """Context manager for the import lock."""
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1420
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1421 def __enter__(self):
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1422 """Acquire the import lock."""
76331 d777f854a66e Issue #13959: Rename imp to _imp and add Lib/imp.py and begin Brett Cannon <brett@python.org> parents: 76330 diff changeset	1423 _imp.acquire_lock()
52141 f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1424
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1425 def __exit__(self, exc_type, exc_value, exc_traceback):
f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1426 """Release the import lock regardless of any raised exceptions."""
76331 d777f854a66e Issue #13959: Rename imp to _imp and add Lib/imp.py and begin Brett Cannon <brett@python.org> parents: 76330 diff changeset	1427 _imp.release_lock()
52141 f188dd68620e Last big re-organization of importlib._bootstrap. Should actually be able to find something in the file now. Brett Cannon <bcannon@gmail.com> parents: 52140 diff changeset	1428
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	1429
74983 03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1430 def _resolve_name(name, package, level):
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1431 """Resolve a relative module name to an absolute one."""
75250 8840037a9c53 unused imports, pep8 Philip Jenvey <pjenvey@underboss.org> parents: 75239 diff changeset	1432 bits = package.rsplit('.', level - 1)
75239 5cfc9c97af23 Simplify importib._resolve_name(). Brett Cannon <brett@python.org> parents: 75235 diff changeset	1433 if len(bits) < level:
5cfc9c97af23 Simplify importib._resolve_name(). Brett Cannon <brett@python.org> parents: 75235 diff changeset	1434 raise ValueError('attempted relative import beyond top-level package')
5cfc9c97af23 Simplify importib._resolve_name(). Brett Cannon <brett@python.org> parents: 75235 diff changeset	1435 base = bits[0]
77964 37850f083af5 Minor refactoring in importlib._bootstrap, and fix the '_wrap' docstring. Florent Xicluna <florent.xicluna@gmail.com> parents: 77917 diff changeset	1436 return '{}.{}'.format(base, name) if name else base
74983 03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1437
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1438
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1439 def _find_module(name, path):
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1440 """Find a module's loader."""
76579 3bd60cc27664 Issue #14605: Stop having implicit entries for sys.meta_path. Brett Cannon <brett@python.org> parents: 76566 diff changeset	1441 if not sys.meta_path:
3bd60cc27664 Issue #14605: Stop having implicit entries for sys.meta_path. Brett Cannon <brett@python.org> parents: 76566 diff changeset	1442 _warnings.warn('sys.meta_path is empty', ImportWarning)
3bd60cc27664 Issue #14605: Stop having implicit entries for sys.meta_path. Brett Cannon <brett@python.org> parents: 76566 diff changeset	1443 for finder in sys.meta_path:
77013 edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1444 with _ImportLockContext():
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1445 loader = finder.find_module(name, path)
74983 03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1446 if loader is not None:
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1447 # The parent import may have already imported this module.
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1448 if name not in sys.modules:
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1449 return loader
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1450 else:
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1451 return sys.modules[name].__loader__
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1452 else:
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1453 return None
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1454
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1455
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1456 def _sanity_check(name, package, level):
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1457 """Verify arguments are "sane"."""
75228 909935a236e3 Do a type check instead of an interface check. Brett Cannon <brett@python.org> parents: 75193 diff changeset	1458 if not isinstance(name, str):
75193 f95faebf5dea Refactor importlib to make it easier to re-implement in C. Brett Cannon <brett@python.org> parents: 75185 diff changeset	1459 raise TypeError("module name must be str, not {}".format(type(name)))
f95faebf5dea Refactor importlib to make it easier to re-implement in C. Brett Cannon <brett@python.org> parents: 75185 diff changeset	1460 if level < 0:
f95faebf5dea Refactor importlib to make it easier to re-implement in C. Brett Cannon <brett@python.org> parents: 75185 diff changeset	1461 raise ValueError('level must be >= 0')
74983 03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1462 if package:
75228 909935a236e3 Do a type check instead of an interface check. Brett Cannon <brett@python.org> parents: 75193 diff changeset	1463 if not isinstance(package, str):
75229 8d1040fdac60 Improper type for __package__ should raise TypeError, not ValueError. Brett Cannon <brett@python.org> parents: 75228 diff changeset	1464 raise TypeError("__package__ not set to a string")
74983 03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1465 elif package not in sys.modules:
77964 37850f083af5 Minor refactoring in importlib._bootstrap, and fix the '_wrap' docstring. Florent Xicluna <florent.xicluna@gmail.com> parents: 77917 diff changeset	1466 msg = ("Parent module {!r} not loaded, cannot perform relative "
74983 03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1467 "import")
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1468 raise SystemError(msg.format(package))
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1469 if not name and level == 0:
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1470 raise ValueError("Empty module name")
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1471
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1472
75193 f95faebf5dea Refactor importlib to make it easier to re-implement in C. Brett Cannon <brett@python.org> parents: 75185 diff changeset	1473 _ERR_MSG = 'No module named {!r}'
74983 03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1474
77013 edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1475 def _find_and_load_unlocked(name, import_):
74983 03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1476 path = None
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1477 parent = name.rpartition('.')[0]
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1478 if parent:
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1479 if parent not in sys.modules:
78344 62033490ca0f Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise Nick Coghlan <ncoghlan@gmail.com> parents: 78314 diff changeset	1480 _call_with_frames_removed(import_, parent)
76081 a40cd5976215 If a module injects something into sys.modules as a side-effect of Brett Cannon <brett@python.org> parents: 75463 diff changeset	1481 # Crazy side-effects!
a40cd5976215 If a module injects something into sys.modules as a side-effect of Brett Cannon <brett@python.org> parents: 75463 diff changeset	1482 if name in sys.modules:
a40cd5976215 If a module injects something into sys.modules as a side-effect of Brett Cannon <brett@python.org> parents: 75463 diff changeset	1483 return sys.modules[name]
74983 03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1484 # Backwards-compatibility; be nicer to skip the dict lookup.
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1485 parent_module = sys.modules[parent]
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1486 try:
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1487 path = parent_module.__path__
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1488 except AttributeError:
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1489 msg = (_ERR_MSG + '; {} is not a package').format(name, parent)
76280 c071af95772a Have importlib take advantage of ImportError's new 'name' and 'path' Brett Cannon <brett@python.org> parents: 76081 diff changeset	1490 raise ImportError(msg, name=name)
75193 f95faebf5dea Refactor importlib to make it easier to re-implement in C. Brett Cannon <brett@python.org> parents: 75185 diff changeset	1491 loader = _find_module(name, path)
f95faebf5dea Refactor importlib to make it easier to re-implement in C. Brett Cannon <brett@python.org> parents: 75185 diff changeset	1492 if loader is None:
76280 c071af95772a Have importlib take advantage of ImportError's new 'name' and 'path' Brett Cannon <brett@python.org> parents: 76081 diff changeset	1493 raise ImportError(_ERR_MSG.format(name), name=name)
76390 db5e3431ee4c rollback 005fd1fe31ab (see #14609 and #14582) Benjamin Peterson <benjamin@python.org> parents: 76384 diff changeset	1494 elif name not in sys.modules:
db5e3431ee4c rollback 005fd1fe31ab (see #14609 and #14582) Benjamin Peterson <benjamin@python.org> parents: 76384 diff changeset	1495 # The parent import may have already imported this module.
db5e3431ee4c rollback 005fd1fe31ab (see #14609 and #14582) Benjamin Peterson <benjamin@python.org> parents: 76384 diff changeset	1496 loader.load_module(name)
76753 59b8a7fcb047 Properly mark names in importlib._bootstrap as private. Brett Cannon <brett@python.org> parents: 76586 diff changeset	1497 _verbose_message('import {!r} # {!r}', name, loader)
76390 db5e3431ee4c rollback 005fd1fe31ab (see #14609 and #14582) Benjamin Peterson <benjamin@python.org> parents: 76384 diff changeset	1498 # Backwards-compatibility; be nicer to skip the dict lookup.
db5e3431ee4c rollback 005fd1fe31ab (see #14609 and #14582) Benjamin Peterson <benjamin@python.org> parents: 76384 diff changeset	1499 module = sys.modules[name]
75193 f95faebf5dea Refactor importlib to make it easier to re-implement in C. Brett Cannon <brett@python.org> parents: 75185 diff changeset	1500 if parent:
f95faebf5dea Refactor importlib to make it easier to re-implement in C. Brett Cannon <brett@python.org> parents: 75185 diff changeset	1501 # Set the module as an attribute on its parent.
f95faebf5dea Refactor importlib to make it easier to re-implement in C. Brett Cannon <brett@python.org> parents: 75185 diff changeset	1502 parent_module = sys.modules[parent]
f95faebf5dea Refactor importlib to make it easier to re-implement in C. Brett Cannon <brett@python.org> parents: 75185 diff changeset	1503 setattr(parent_module, name.rpartition('.')[2], module)
f95faebf5dea Refactor importlib to make it easier to re-implement in C. Brett Cannon <brett@python.org> parents: 75185 diff changeset	1504 # Set __package__ if the loader did not.
77964 37850f083af5 Minor refactoring in importlib._bootstrap, and fix the '_wrap' docstring. Florent Xicluna <florent.xicluna@gmail.com> parents: 77917 diff changeset	1505 if getattr(module, '__package__', None) is None:
75193 f95faebf5dea Refactor importlib to make it easier to re-implement in C. Brett Cannon <brett@python.org> parents: 75185 diff changeset	1506 try:
f95faebf5dea Refactor importlib to make it easier to re-implement in C. Brett Cannon <brett@python.org> parents: 75185 diff changeset	1507 module.__package__ = module.__name__
f95faebf5dea Refactor importlib to make it easier to re-implement in C. Brett Cannon <brett@python.org> parents: 75185 diff changeset	1508 if not hasattr(module, '__path__'):
f95faebf5dea Refactor importlib to make it easier to re-implement in C. Brett Cannon <brett@python.org> parents: 75185 diff changeset	1509 module.__package__ = module.__package__.rpartition('.')[0]
f95faebf5dea Refactor importlib to make it easier to re-implement in C. Brett Cannon <brett@python.org> parents: 75185 diff changeset	1510 except AttributeError:
f95faebf5dea Refactor importlib to make it easier to re-implement in C. Brett Cannon <brett@python.org> parents: 75185 diff changeset	1511 pass
76586 496c68f90a03 Issue #14646: __import__() now sets __loader__ if need be. Brett Cannon <brett@python.org> parents: 76582 diff changeset	1512 # Set loader if need be.
496c68f90a03 Issue #14646: __import__() now sets __loader__ if need be. Brett Cannon <brett@python.org> parents: 76582 diff changeset	1513 if not hasattr(module, '__loader__'):
496c68f90a03 Issue #14646: __import__() now sets __loader__ if need be. Brett Cannon <brett@python.org> parents: 76582 diff changeset	1514 try:
496c68f90a03 Issue #14646: __import__() now sets __loader__ if need be. Brett Cannon <brett@python.org> parents: 76582 diff changeset	1515 module.__loader__ = loader
496c68f90a03 Issue #14646: __import__() now sets __loader__ if need be. Brett Cannon <brett@python.org> parents: 76582 diff changeset	1516 except AttributeError:
496c68f90a03 Issue #14646: __import__() now sets __loader__ if need be. Brett Cannon <brett@python.org> parents: 76582 diff changeset	1517 pass
75193 f95faebf5dea Refactor importlib to make it easier to re-implement in C. Brett Cannon <brett@python.org> parents: 75185 diff changeset	1518 return module
74983 03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1519
66226 af13fb6fb3df Fix a minor inconsistency in capitalization for the 'No module named' exception Brett Cannon <bcannon@gmail.com> parents: 64407 diff changeset	1520
77013 edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1521 def _find_and_load(name, import_):
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1522 """Find and load the module, and release the import lock."""
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1523 try:
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1524 lock = _get_module_lock(name)
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1525 finally:
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1526 _imp.release_lock()
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1527 lock.acquire()
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1528 try:
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1529 return _find_and_load_unlocked(name, import_)
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1530 finally:
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1531 lock.release()
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1532
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1533
51383 a00e5dcea16c Initial, untested stab at writing a common denominator function for __import__ Brett Cannon <bcannon@gmail.com> parents: 51356 diff changeset	1534 def _gcd_import(name, package=None, level=0):
a00e5dcea16c Initial, untested stab at writing a common denominator function for __import__ Brett Cannon <bcannon@gmail.com> parents: 51356 diff changeset	1535 """Import and return the module based on its name, the package the call is
a00e5dcea16c Initial, untested stab at writing a common denominator function for __import__ Brett Cannon <bcannon@gmail.com> parents: 51356 diff changeset	1536 being made from, and the level adjustment.
a00e5dcea16c Initial, untested stab at writing a common denominator function for __import__ Brett Cannon <bcannon@gmail.com> parents: 51356 diff changeset	1537
a00e5dcea16c Initial, untested stab at writing a common denominator function for __import__ Brett Cannon <bcannon@gmail.com> parents: 51356 diff changeset	1538 This function represents the greatest common denominator of functionality
68447 c919a82b5a40 Typos. Eric V. Smith <eric@trueblade.com> parents: 68257 diff changeset	1539 between import_module and __import__. This includes setting __package__ if
51426 7a1356d2aeef Rewrite the code implementing __import__ for importlib. Now it is much simpler Brett Cannon <bcannon@gmail.com> parents: 51383 diff changeset	1540 the loader did not.
7a1356d2aeef Rewrite the code implementing __import__ for importlib. Now it is much simpler Brett Cannon <bcannon@gmail.com> parents: 51383 diff changeset	1541
51383 a00e5dcea16c Initial, untested stab at writing a common denominator function for __import__ Brett Cannon <bcannon@gmail.com> parents: 51356 diff changeset	1542 """
74983 03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1543 _sanity_check(name, package, level)
51383 a00e5dcea16c Initial, untested stab at writing a common denominator function for __import__ Brett Cannon <bcannon@gmail.com> parents: 51356 diff changeset	1544 if level > 0:
74983 03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1545 name = _resolve_name(name, package, level)
77013 edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1546 _imp.acquire_lock()
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1547 if name not in sys.modules:
75193 f95faebf5dea Refactor importlib to make it easier to re-implement in C. Brett Cannon <brett@python.org> parents: 75185 diff changeset	1548 return _find_and_load(name, _gcd_import)
77013 edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1549 module = sys.modules[name]
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1550 if module is None:
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1551 _imp.release_lock()
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1552 message = ("import of {} halted; "
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1553 "None in sys.modules".format(name))
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1554 raise ImportError(message, name=name)
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1555 _lock_unlock_module(name)
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1556 return module
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	1557
75235 3670df23081d Turn _return_module() into _handle_fromlist(). Brett Cannon <brett@python.org> parents: 75229 diff changeset	1558 def _handle_fromlist(module, fromlist, import_):
74983 03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1559 """Figure out what __import__ should return.
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	1560
74983 03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1561 The import_ parameter is a callable which takes the name of module to
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1562 import. It is required to decouple the function from assuming importlib's
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1563 import implementation is desired.
50845 aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is Brett Cannon <bcannon@gmail.com> parents: diff changeset	1564
51426 7a1356d2aeef Rewrite the code implementing __import__ for importlib. Now it is much simpler Brett Cannon <bcannon@gmail.com> parents: 51383 diff changeset	1565 """
7a1356d2aeef Rewrite the code implementing __import__ for importlib. Now it is much simpler Brett Cannon <bcannon@gmail.com> parents: 51383 diff changeset	1566 # The hell that is fromlist ...
75235 3670df23081d Turn _return_module() into _handle_fromlist(). Brett Cannon <brett@python.org> parents: 75229 diff changeset	1567 # If a package was imported, try to import stuff from fromlist.
3670df23081d Turn _return_module() into _handle_fromlist(). Brett Cannon <brett@python.org> parents: 75229 diff changeset	1568 if hasattr(module, '__path__'):
78051 dc18a2a66d16 Issue #15111: When a module was imported using a 'from import' Brett Cannon <brett@python.org> parents: 78042 diff changeset	1569 if '*' in fromlist:
75235 3670df23081d Turn _return_module() into _handle_fromlist(). Brett Cannon <brett@python.org> parents: 75229 diff changeset	1570 fromlist = list(fromlist)
3670df23081d Turn _return_module() into _handle_fromlist(). Brett Cannon <brett@python.org> parents: 75229 diff changeset	1571 fromlist.remove('*')
78051 dc18a2a66d16 Issue #15111: When a module was imported using a 'from import' Brett Cannon <brett@python.org> parents: 78042 diff changeset	1572 if hasattr(module, '__all__'):
dc18a2a66d16 Issue #15111: When a module was imported using a 'from import' Brett Cannon <brett@python.org> parents: 78042 diff changeset	1573 fromlist.extend(module.__all__)
77964 37850f083af5 Minor refactoring in importlib._bootstrap, and fix the '_wrap' docstring. Florent Xicluna <florent.xicluna@gmail.com> parents: 77917 diff changeset	1574 for x in fromlist:
37850f083af5 Minor refactoring in importlib._bootstrap, and fix the '_wrap' docstring. Florent Xicluna <florent.xicluna@gmail.com> parents: 77917 diff changeset	1575 if not hasattr(module, x):
78344 62033490ca0f Close #15486: Simplify the mechanism used to remove importlib frames from tracebacks when they just introduce irrelevant noise Nick Coghlan <ncoghlan@gmail.com> parents: 78314 diff changeset	1576 _call_with_frames_removed(import_,
78314 75a30a478dc7 Close #15425: Eliminate more importlib related traceback noise Nick Coghlan <ncoghlan@gmail.com> parents: 78308 diff changeset	1577 '{}.{}'.format(module.__name__, x))
75235 3670df23081d Turn _return_module() into _handle_fromlist(). Brett Cannon <brett@python.org> parents: 75229 diff changeset	1578 return module
74839 322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1579
322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1580
74983 03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1581 def _calc___package__(globals):
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1582 """Calculate what __package__ should be.
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1583
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1584 __package__ is not guaranteed to be defined or could be set to None
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1585 to represent that its proper value is unknown.
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1586
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1587 """
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1588 package = globals.get('__package__')
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1589 if package is None:
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1590 package = globals['__name__']
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1591 if '__path__' not in globals:
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1592 package = package.rpartition('.')[0]
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1593 return package
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1594
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1595
78308 bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	1596 def _get_supported_file_loaders():
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	1597 """Returns a list of file-based module loaders.
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	1598
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	1599 Each item is a tuple (loader, suffixes, allow_packages).
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	1600 """
78486 1db6553f3f8c Issue #15576: Allow extension modules to be a package's __init__ Brett Cannon <brett@python.org> parents: 78485 diff changeset	1601 extensions = ExtensionFileLoader, _imp.extension_suffixes()
1db6553f3f8c Issue #15576: Allow extension modules to be a package's __init__ Brett Cannon <brett@python.org> parents: 78485 diff changeset	1602 source = SourceFileLoader, SOURCE_SUFFIXES
1db6553f3f8c Issue #15576: Allow extension modules to be a package's __init__ Brett Cannon <brett@python.org> parents: 78485 diff changeset	1603 bytecode = SourcelessFileLoader, BYTECODE_SUFFIXES
78308 bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	1604 return [extensions, source, bytecode]
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	1605
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	1606
78456 4240282a9f4a Issue #15471: Don't use mutable object as default values for the Brett Cannon <brett@python.org> parents: 78382 diff changeset	1607 def __import__(name, globals=None, locals=None, fromlist=(), level=0):
74983 03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1608 """Import a module.
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1609
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1610 The 'globals' argument is used to infer where the import is occuring from
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1611 to handle relative imports. The 'locals' argument is ignored. The
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1612 'fromlist' argument specifies what should exist as attributes on the module
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1613 being imported (e.g. ``from module import <fromlist>``). The 'level'
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1614 argument represents the package location to import from in a relative
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1615 import (e.g. ``from ..pkg import mod`` would have a 'level' of 2).
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1616
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1617 """
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1618 if level == 0:
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1619 module = _gcd_import(name)
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1620 else:
78456 4240282a9f4a Issue #15471: Don't use mutable object as default values for the Brett Cannon <brett@python.org> parents: 78382 diff changeset	1621 globals_ = globals if globals is not None else {}
4240282a9f4a Issue #15471: Don't use mutable object as default values for the Brett Cannon <brett@python.org> parents: 78382 diff changeset	1622 package = _calc___package__(globals_)
74983 03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1623 module = _gcd_import(name, package, level)
75235 3670df23081d Turn _return_module() into _handle_fromlist(). Brett Cannon <brett@python.org> parents: 75229 diff changeset	1624 if not fromlist:
3670df23081d Turn _return_module() into _handle_fromlist(). Brett Cannon <brett@python.org> parents: 75229 diff changeset	1625 # Return up to the first dot in 'name'. This is complicated by the fact
3670df23081d Turn _return_module() into _handle_fromlist(). Brett Cannon <brett@python.org> parents: 75229 diff changeset	1626 # that 'name' may be relative.
3670df23081d Turn _return_module() into _handle_fromlist(). Brett Cannon <brett@python.org> parents: 75229 diff changeset	1627 if level == 0:
76826 d6324941b739 Issue #14583: Fix importlib bug when a package's __init__.py would first import one of its modules then raise an error. Antoine Pitrou <solipsis@pitrou.net> parents: 76765 diff changeset	1628 return _gcd_import(name.partition('.')[0])
75235 3670df23081d Turn _return_module() into _handle_fromlist(). Brett Cannon <brett@python.org> parents: 75229 diff changeset	1629 elif not name:
3670df23081d Turn _return_module() into _handle_fromlist(). Brett Cannon <brett@python.org> parents: 75229 diff changeset	1630 return module
3670df23081d Turn _return_module() into _handle_fromlist(). Brett Cannon <brett@python.org> parents: 75229 diff changeset	1631 else:
3670df23081d Turn _return_module() into _handle_fromlist(). Brett Cannon <brett@python.org> parents: 75229 diff changeset	1632 cut_off = len(name) - len(name.partition('.')[0])
76312 9e8cbf07068a Handle importing pkg.mod by executing Brett Cannon <brett@python.org> parents: 76309 diff changeset	1633 return sys.modules[module.__name__[:len(module.__name__)-cut_off]]
75235 3670df23081d Turn _return_module() into _handle_fromlist(). Brett Cannon <brett@python.org> parents: 75229 diff changeset	1634 else:
3670df23081d Turn _return_module() into _handle_fromlist(). Brett Cannon <brett@python.org> parents: 75229 diff changeset	1635 return _handle_fromlist(module, fromlist, _gcd_import)
74983 03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1636
03140936913c Refactor importlib.__import__() and _gcd_import() to facilitate using Brett Cannon <brett@python.org> parents: 74935 diff changeset	1637
76445 c820aa9c0c00 Issue #13959: Keep imp.get_magic() in C code, but cache in importlib Brett Cannon <brett@python.org> parents: 76428 diff changeset	1638
76331 d777f854a66e Issue #13959: Rename imp to _imp and add Lib/imp.py and begin Brett Cannon <brett@python.org> parents: 76330 diff changeset	1639 def _setup(sys_module, _imp_module):
74839 322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1640 """Setup importlib by importing needed built-in modules and injecting them
322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1641 into the global namespace.
322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1642
76331 d777f854a66e Issue #13959: Rename imp to _imp and add Lib/imp.py and begin Brett Cannon <brett@python.org> parents: 76330 diff changeset	1643 As sys is needed for sys.modules access and _imp is needed to load built-in
74935 81f3bb5efa40 Bring importlib in line w/ changes made in my personal bootstrap branch in the sandbox. Brett Cannon <brett@python.org> parents: 74867 diff changeset	1644 modules, those two modules must be explicitly passed in.
74839 322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1645
322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1646 """
76331 d777f854a66e Issue #13959: Rename imp to _imp and add Lib/imp.py and begin Brett Cannon <brett@python.org> parents: 76330 diff changeset	1647 global _imp, sys
d777f854a66e Issue #13959: Rename imp to _imp and add Lib/imp.py and begin Brett Cannon <brett@python.org> parents: 76330 diff changeset	1648 _imp = _imp_module
74839 322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1649 sys = sys_module
322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1650
76331 d777f854a66e Issue #13959: Rename imp to _imp and add Lib/imp.py and begin Brett Cannon <brett@python.org> parents: 76330 diff changeset	1651 for module in (_imp, sys):
74839 322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1652 if not hasattr(module, '__loader__'):
322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1653 module.__loader__ = BuiltinImporter
322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1654
322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1655 self_module = sys.modules[__name__]
322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1656 for builtin_name in ('_io', '_warnings', 'builtins', 'marshal'):
322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1657 if builtin_name not in sys.modules:
322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1658 builtin_module = BuiltinImporter.load_module(builtin_name)
322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1659 else:
322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1660 builtin_module = sys.modules[builtin_name]
322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1661 setattr(self_module, builtin_name, builtin_module)
322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1662
76449 163360370ad2 Make path manipulation more robust for platforms with alternative path Brett Cannon <brett@python.org> parents: 76445 diff changeset	1663 os_details = ('posix', ['/']), ('nt', ['\\', '/']), ('os2', ['\\', '/'])
163360370ad2 Make path manipulation more robust for platforms with alternative path Brett Cannon <brett@python.org> parents: 76445 diff changeset	1664 for builtin_os, path_separators in os_details:
76472 4e853913054c Issue #13959: Continue to try to accomodate altsep in importlib by not Brett Cannon <brett@python.org> parents: 76466 diff changeset	1665 # Assumption made in _path_join()
4e853913054c Issue #13959: Continue to try to accomodate altsep in importlib by not Brett Cannon <brett@python.org> parents: 76466 diff changeset	1666 assert all(len(sep) == 1 for sep in path_separators)
76449 163360370ad2 Make path manipulation more robust for platforms with alternative path Brett Cannon <brett@python.org> parents: 76445 diff changeset	1667 path_sep = path_separators[0]
74839 322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1668 if builtin_os in sys.modules:
322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1669 os_module = sys.modules[builtin_os]
322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1670 break
322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1671 else:
322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1672 try:
322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1673 os_module = BuiltinImporter.load_module(builtin_os)
322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1674 # TODO: rip out os2 code after 3.3 is released as per PEP 11
322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1675 if builtin_os == 'os2' and 'EMX GCC' in sys.version:
76449 163360370ad2 Make path manipulation more robust for platforms with alternative path Brett Cannon <brett@python.org> parents: 76445 diff changeset	1676 path_sep = path_separators[1]
74839 322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1677 break
322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1678 except ImportError:
322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1679 continue
322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1680 else:
322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1681 raise ImportError('importlib requires posix or nt')
77013 edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1682
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1683 try:
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1684 thread_module = BuiltinImporter.load_module('_thread')
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1685 except ImportError:
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1686 # Python was built without threads
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1687 thread_module = None
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1688 weakref_module = BuiltinImporter.load_module('_weakref')
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1689
78308 bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	1690 if builtin_os == 'nt':
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	1691 winreg_module = BuiltinImporter.load_module('winreg')
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	1692 setattr(self_module, '_winreg', winreg_module)
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	1693
74839 322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1694 setattr(self_module, '_os', os_module)
77013 edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1695 setattr(self_module, '_thread', thread_module)
edb9ce3a6c2e Issue #9260: A finer-grained import lock. Antoine Pitrou <solipsis@pitrou.net> parents: 76877 diff changeset	1696 setattr(self_module, '_weakref', weakref_module)
74839 322e6f818cd2 Move setup code from importlib.__init__ to Brett Cannon <brett@python.org> parents: 74838 diff changeset	1697 setattr(self_module, 'path_sep', path_sep)
76449 163360370ad2 Make path manipulation more robust for platforms with alternative path Brett Cannon <brett@python.org> parents: 76445 diff changeset	1698 setattr(self_module, 'path_separators', set(path_separators))
75048 bbaab666e6c7 Issue #14043: Speed up importlib's _FileFinder by at least 8x, and add a new importlib.invalidate_caches() function. Antoine Pitrou <solipsis@pitrou.net> parents: 75045 diff changeset	1699 # Constants
75055 b8576719fe71 _relax_case -> _make_relax_case Antoine Pitrou <solipsis@pitrou.net> parents: 75048 diff changeset	1700 setattr(self_module, '_relax_case', _make_relax_case())
78486 1db6553f3f8c Issue #15576: Allow extension modules to be a package's __init__ Brett Cannon <brett@python.org> parents: 78485 diff changeset	1701 EXTENSION_SUFFIXES.extend(_imp.extension_suffixes())
76451 ea46ebba8a0f Issue #13959: Re-implement imp.source_from_cache() in Lib/imp.py. Brett Cannon <brett@python.org> parents: 76450 diff changeset	1702 if builtin_os == 'nt':
76874 b81ddaf0db47 Issue #13959: Deprecate imp.get_suffixes() for new attributes on Brett Cannon <brett@python.org> parents: 76826 diff changeset	1703 SOURCE_SUFFIXES.append('.pyw')
78486 1db6553f3f8c Issue #15576: Allow extension modules to be a package's __init__ Brett Cannon <brett@python.org> parents: 78485 diff changeset	1704 if '_d.pyd' in EXTENSION_SUFFIXES:
78380 a1ac1e13c5a0 Close #15519: Properly expose WindowsRegistryFinder in importlib and bring the name into line with normal import terminology. Original patch by Eric Snow Nick Coghlan <ncoghlan@gmail.com> parents: 78344 diff changeset	1705 WindowsRegistryFinder.DEBUG_BUILD = True
75000 e477832212b9 Optimize importlib's case-sensitivity check by wasting as little time as possible under case-sensitive OSs. Brett Cannon <brett@python.org> parents: 74998 diff changeset	1706
74935 81f3bb5efa40 Bring importlib in line w/ changes made in my personal bootstrap branch in the sandbox. Brett Cannon <brett@python.org> parents: 74867 diff changeset	1707
76331 d777f854a66e Issue #13959: Rename imp to _imp and add Lib/imp.py and begin Brett Cannon <brett@python.org> parents: 76330 diff changeset	1708 def _install(sys_module, _imp_module):
76765 146001e3364d Clean up a docstring. Brett Cannon <brett@python.org> parents: 76755 diff changeset	1709 """Install importlib as the implementation of import."""
76331 d777f854a66e Issue #13959: Rename imp to _imp and add Lib/imp.py and begin Brett Cannon <brett@python.org> parents: 76330 diff changeset	1710 _setup(sys_module, _imp_module)
78308 bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	1711 supported_loaders = _get_supported_file_loaders()
76582 7025ee00dbf6 Issue #14605: Use None in sys.path_importer_cache to represent no Brett Cannon <brett@python.org> parents: 76579 diff changeset	1712 sys.path_hooks.extend([FileFinder.path_hook(*supported_loaders)])
78308 bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	1713 sys.meta_path.append(BuiltinImporter)
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	1714 sys.meta_path.append(FrozenImporter)
bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	1715 if _os.__name__ == 'nt':
78380 a1ac1e13c5a0 Close #15519: Properly expose WindowsRegistryFinder in importlib and bring the name into line with normal import terminology. Original patch by Eric Snow Nick Coghlan <ncoghlan@gmail.com> parents: 78344 diff changeset	1716 sys.meta_path.append(WindowsRegistryFinder)
78308 bd58c421057c Issue #14578: Support modules registered in the Windows registry again. Martin v. Löwis <martin@v.loewis.de> parents: 78112 diff changeset	1717 sys.meta_path.append(PathFinder)

Mercurial > cpython-lru_cache-weakref

annotate Lib/importlib/_bootstrap.py @ 78489:e7a67f1bf604