cpython-lru_cache-weakref: Lib/pkgutil.py annotate

annotate Lib/pkgutil.py @ 86127:a3d09c19bb93

merge with 3.3

author	Georg Brandl <georg@python.org>
date	Sun, 06 Oct 2013 19:24:00 +0200
parents	1b8f08c4efd5
children	0c0cbba64b7b

rev	line source
26753 7c15bf7726b2 Last week we discussed adding this module to the standard library. Guido van Rossum <guido@python.org> parents: diff changeset	1 """Utilities to support packages."""
7c15bf7726b2 Last week we discussed adding this module to the standard library. Guido van Rossum <guido@python.org> parents: diff changeset	2
84039 dfcb64f51f7b Add reference implementation for PEP 443 Łukasz Langa <lukasz@langa.pl> parents: 83367 diff changeset	3 from functools import singledispatch as simplegeneric
dfcb64f51f7b Add reference implementation for PEP 443 Łukasz Langa <lukasz@langa.pl> parents: 83367 diff changeset	4 import importlib
84167 1b8f08c4efd5 Issue #17177: The imp module is pending deprecation. Brett Cannon <brett@python.org> parents: 84039 diff changeset	5 import importlib.util
84039 dfcb64f51f7b Add reference implementation for PEP 443 Łukasz Langa <lukasz@langa.pl> parents: 83367 diff changeset	6 import os
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	7 import os.path
84039 dfcb64f51f7b Add reference implementation for PEP 443 Łukasz Langa <lukasz@langa.pl> parents: 83367 diff changeset	8 import sys
dfcb64f51f7b Add reference implementation for PEP 443 Łukasz Langa <lukasz@langa.pl> parents: 83367 diff changeset	9 from types import ModuleType
84167 1b8f08c4efd5 Issue #17177: The imp module is pending deprecation. Brett Cannon <brett@python.org> parents: 84039 diff changeset	10 import warnings
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	11
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	12 __all__ = [
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	13 'get_importer', 'iter_importers', 'get_loader', 'find_loader',
70406 2ee64c205d37 Add missing name in pkgutil.__all__ Éric Araujo <merwok@netwok.org> parents: 55602 diff changeset	14 'walk_packages', 'iter_modules', 'get_data',
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	15 'ImpImporter', 'ImpLoader', 'read_code', 'extend_path',
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	16 ]
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	17
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	18 def read_code(stream):
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	19 # This helper is needed in order for the PEP 302 emulation to
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	20 # correctly handle compiled files
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	21 import marshal
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	22
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	23 magic = stream.read(4)
84167 1b8f08c4efd5 Issue #17177: The imp module is pending deprecation. Brett Cannon <brett@python.org> parents: 84039 diff changeset	24 if magic != importlib.util.MAGIC_NUMBER:
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	25 return None
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	26
74358 87331661042b Issue #13645: pyc files now contain the size of the corresponding source Antoine Pitrou <solipsis@pitrou.net> parents: 72775 diff changeset	27 stream.read(8) # Skip timestamp and size
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	28 return marshal.load(stream)
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	29
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	30
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	31 def walk_packages(path=None, prefix='', onerror=None):
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	32 """Yields (module_loader, name, ispkg) for all modules recursively
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	33 on path, or, if path is None, all accessible modules.
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	34
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	35 'path' should be either None or a list of paths to look for
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	36 modules in.
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	37
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	38 'prefix' is a string to output on the front of every module name
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	39 on output.
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	40
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	41 Note that this function must import all packages (NOT all
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	42 modules!) on the given path, in order to access the __path__
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	43 attribute to find submodules.
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	44
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	45 'onerror' is a function which gets called with one argument (the
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	46 name of the package which was being imported) if any exception
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	47 occurs while trying to import a package. If no onerror function is
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	48 supplied, ImportErrors are caught and ignored, while all other
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	49 exceptions are propagated, terminating the search.
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	50
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	51 Examples:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	52
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	53 # list all modules python can access
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	54 walk_packages()
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	55
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	56 # list all submodules of ctypes
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	57 walk_packages(ctypes.__path__, ctypes.__name__+'.')
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	58 """
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	59
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 37879 diff changeset	60 def seen(p, m={}):
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 37879 diff changeset	61 if p in m:
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 37879 diff changeset	62 return True
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	63 m[p] = True
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	64
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	65 for importer, name, ispkg in iter_modules(path, prefix):
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	66 yield importer, name, ispkg
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	67
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	68 if ispkg:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	69 try:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	70 __import__(name)
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	71 except ImportError:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	72 if onerror is not None:
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	73 onerror(name)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	74 except Exception:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	75 if onerror is not None:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	76 onerror(name)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	77 else:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	78 raise
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	79 else:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	80 path = getattr(sys.modules[name], '__path__', None) or []
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	81
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	82 # don't traverse path items we've seen before
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	83 path = [p for p in path if not seen(p)]
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	84
79378 fb90e2ff95b7 utilize yield from Philip Jenvey <pjenvey@underboss.org> parents: 78195 diff changeset	85 yield from walk_packages(path, name+'.', onerror)
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	86
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	87
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	88 def iter_modules(path=None, prefix=''):
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	89 """Yields (module_loader, name, ispkg) for all submodules on path,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	90 or, if path is None, all top-level modules on sys.path.
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	91
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	92 'path' should be either None or a list of paths to look for
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	93 modules in.
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	94
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	95 'prefix' is a string to output on the front of every module name
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	96 on output.
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	97 """
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	98
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	99 if path is None:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	100 importers = iter_importers()
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	101 else:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	102 importers = map(get_importer, path)
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	103
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	104 yielded = {}
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	105 for i in importers:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	106 for name, ispkg in iter_importer_modules(i, prefix):
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	107 if name not in yielded:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	108 yielded[name] = 1
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	109 yield i, name, ispkg
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	110
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	111
84039 dfcb64f51f7b Add reference implementation for PEP 443 Łukasz Langa <lukasz@langa.pl> parents: 83367 diff changeset	112 @simplegeneric
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	113 def iter_importer_modules(importer, prefix=''):
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 37879 diff changeset	114 if not hasattr(importer, 'iter_modules'):
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	115 return []
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	116 return importer.iter_modules(prefix)
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	117
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	118
78111 9101eab6178c Issue #15343: Handle importlib.machinery.FileFinder instances in pkgutil.walk_packages (et al) Nick Coghlan <ncoghlan@gmail.com> parents: 78107 diff changeset	119 # Implement a file walker for the normal importlib path hook
9101eab6178c Issue #15343: Handle importlib.machinery.FileFinder instances in pkgutil.walk_packages (et al) Nick Coghlan <ncoghlan@gmail.com> parents: 78107 diff changeset	120 def _iter_file_finder_modules(importer, prefix=''):
9101eab6178c Issue #15343: Handle importlib.machinery.FileFinder instances in pkgutil.walk_packages (et al) Nick Coghlan <ncoghlan@gmail.com> parents: 78107 diff changeset	121 if importer.path is None or not os.path.isdir(importer.path):
9101eab6178c Issue #15343: Handle importlib.machinery.FileFinder instances in pkgutil.walk_packages (et al) Nick Coghlan <ncoghlan@gmail.com> parents: 78107 diff changeset	122 return
9101eab6178c Issue #15343: Handle importlib.machinery.FileFinder instances in pkgutil.walk_packages (et al) Nick Coghlan <ncoghlan@gmail.com> parents: 78107 diff changeset	123
9101eab6178c Issue #15343: Handle importlib.machinery.FileFinder instances in pkgutil.walk_packages (et al) Nick Coghlan <ncoghlan@gmail.com> parents: 78107 diff changeset	124 yielded = {}
9101eab6178c Issue #15343: Handle importlib.machinery.FileFinder instances in pkgutil.walk_packages (et al) Nick Coghlan <ncoghlan@gmail.com> parents: 78107 diff changeset	125 import inspect
9101eab6178c Issue #15343: Handle importlib.machinery.FileFinder instances in pkgutil.walk_packages (et al) Nick Coghlan <ncoghlan@gmail.com> parents: 78107 diff changeset	126 try:
9101eab6178c Issue #15343: Handle importlib.machinery.FileFinder instances in pkgutil.walk_packages (et al) Nick Coghlan <ncoghlan@gmail.com> parents: 78107 diff changeset	127 filenames = os.listdir(importer.path)
9101eab6178c Issue #15343: Handle importlib.machinery.FileFinder instances in pkgutil.walk_packages (et al) Nick Coghlan <ncoghlan@gmail.com> parents: 78107 diff changeset	128 except OSError:
9101eab6178c Issue #15343: Handle importlib.machinery.FileFinder instances in pkgutil.walk_packages (et al) Nick Coghlan <ncoghlan@gmail.com> parents: 78107 diff changeset	129 # ignore unreadable directories like import does
9101eab6178c Issue #15343: Handle importlib.machinery.FileFinder instances in pkgutil.walk_packages (et al) Nick Coghlan <ncoghlan@gmail.com> parents: 78107 diff changeset	130 filenames = []
9101eab6178c Issue #15343: Handle importlib.machinery.FileFinder instances in pkgutil.walk_packages (et al) Nick Coghlan <ncoghlan@gmail.com> parents: 78107 diff changeset	131 filenames.sort() # handle packages before same-named modules
9101eab6178c Issue #15343: Handle importlib.machinery.FileFinder instances in pkgutil.walk_packages (et al) Nick Coghlan <ncoghlan@gmail.com> parents: 78107 diff changeset	132
9101eab6178c Issue #15343: Handle importlib.machinery.FileFinder instances in pkgutil.walk_packages (et al) Nick Coghlan <ncoghlan@gmail.com> parents: 78107 diff changeset	133 for fn in filenames:
9101eab6178c Issue #15343: Handle importlib.machinery.FileFinder instances in pkgutil.walk_packages (et al) Nick Coghlan <ncoghlan@gmail.com> parents: 78107 diff changeset	134 modname = inspect.getmodulename(fn)
9101eab6178c Issue #15343: Handle importlib.machinery.FileFinder instances in pkgutil.walk_packages (et al) Nick Coghlan <ncoghlan@gmail.com> parents: 78107 diff changeset	135 if modname=='__init__' or modname in yielded:
9101eab6178c Issue #15343: Handle importlib.machinery.FileFinder instances in pkgutil.walk_packages (et al) Nick Coghlan <ncoghlan@gmail.com> parents: 78107 diff changeset	136 continue
9101eab6178c Issue #15343: Handle importlib.machinery.FileFinder instances in pkgutil.walk_packages (et al) Nick Coghlan <ncoghlan@gmail.com> parents: 78107 diff changeset	137
9101eab6178c Issue #15343: Handle importlib.machinery.FileFinder instances in pkgutil.walk_packages (et al) Nick Coghlan <ncoghlan@gmail.com> parents: 78107 diff changeset	138 path = os.path.join(importer.path, fn)
9101eab6178c Issue #15343: Handle importlib.machinery.FileFinder instances in pkgutil.walk_packages (et al) Nick Coghlan <ncoghlan@gmail.com> parents: 78107 diff changeset	139 ispkg = False
9101eab6178c Issue #15343: Handle importlib.machinery.FileFinder instances in pkgutil.walk_packages (et al) Nick Coghlan <ncoghlan@gmail.com> parents: 78107 diff changeset	140
9101eab6178c Issue #15343: Handle importlib.machinery.FileFinder instances in pkgutil.walk_packages (et al) Nick Coghlan <ncoghlan@gmail.com> parents: 78107 diff changeset	141 if not modname and os.path.isdir(path) and '.' not in fn:
9101eab6178c Issue #15343: Handle importlib.machinery.FileFinder instances in pkgutil.walk_packages (et al) Nick Coghlan <ncoghlan@gmail.com> parents: 78107 diff changeset	142 modname = fn
9101eab6178c Issue #15343: Handle importlib.machinery.FileFinder instances in pkgutil.walk_packages (et al) Nick Coghlan <ncoghlan@gmail.com> parents: 78107 diff changeset	143 try:
9101eab6178c Issue #15343: Handle importlib.machinery.FileFinder instances in pkgutil.walk_packages (et al) Nick Coghlan <ncoghlan@gmail.com> parents: 78107 diff changeset	144 dircontents = os.listdir(path)
9101eab6178c Issue #15343: Handle importlib.machinery.FileFinder instances in pkgutil.walk_packages (et al) Nick Coghlan <ncoghlan@gmail.com> parents: 78107 diff changeset	145 except OSError:
9101eab6178c Issue #15343: Handle importlib.machinery.FileFinder instances in pkgutil.walk_packages (et al) Nick Coghlan <ncoghlan@gmail.com> parents: 78107 diff changeset	146 # ignore unreadable directories like import does
9101eab6178c Issue #15343: Handle importlib.machinery.FileFinder instances in pkgutil.walk_packages (et al) Nick Coghlan <ncoghlan@gmail.com> parents: 78107 diff changeset	147 dircontents = []
9101eab6178c Issue #15343: Handle importlib.machinery.FileFinder instances in pkgutil.walk_packages (et al) Nick Coghlan <ncoghlan@gmail.com> parents: 78107 diff changeset	148 for fn in dircontents:
9101eab6178c Issue #15343: Handle importlib.machinery.FileFinder instances in pkgutil.walk_packages (et al) Nick Coghlan <ncoghlan@gmail.com> parents: 78107 diff changeset	149 subname = inspect.getmodulename(fn)
9101eab6178c Issue #15343: Handle importlib.machinery.FileFinder instances in pkgutil.walk_packages (et al) Nick Coghlan <ncoghlan@gmail.com> parents: 78107 diff changeset	150 if subname=='__init__':
9101eab6178c Issue #15343: Handle importlib.machinery.FileFinder instances in pkgutil.walk_packages (et al) Nick Coghlan <ncoghlan@gmail.com> parents: 78107 diff changeset	151 ispkg = True
9101eab6178c Issue #15343: Handle importlib.machinery.FileFinder instances in pkgutil.walk_packages (et al) Nick Coghlan <ncoghlan@gmail.com> parents: 78107 diff changeset	152 break
9101eab6178c Issue #15343: Handle importlib.machinery.FileFinder instances in pkgutil.walk_packages (et al) Nick Coghlan <ncoghlan@gmail.com> parents: 78107 diff changeset	153 else:
9101eab6178c Issue #15343: Handle importlib.machinery.FileFinder instances in pkgutil.walk_packages (et al) Nick Coghlan <ncoghlan@gmail.com> parents: 78107 diff changeset	154 continue # not a package
9101eab6178c Issue #15343: Handle importlib.machinery.FileFinder instances in pkgutil.walk_packages (et al) Nick Coghlan <ncoghlan@gmail.com> parents: 78107 diff changeset	155
9101eab6178c Issue #15343: Handle importlib.machinery.FileFinder instances in pkgutil.walk_packages (et al) Nick Coghlan <ncoghlan@gmail.com> parents: 78107 diff changeset	156 if modname and '.' not in modname:
9101eab6178c Issue #15343: Handle importlib.machinery.FileFinder instances in pkgutil.walk_packages (et al) Nick Coghlan <ncoghlan@gmail.com> parents: 78107 diff changeset	157 yielded[modname] = 1
9101eab6178c Issue #15343: Handle importlib.machinery.FileFinder instances in pkgutil.walk_packages (et al) Nick Coghlan <ncoghlan@gmail.com> parents: 78107 diff changeset	158 yield prefix + modname, ispkg
9101eab6178c Issue #15343: Handle importlib.machinery.FileFinder instances in pkgutil.walk_packages (et al) Nick Coghlan <ncoghlan@gmail.com> parents: 78107 diff changeset	159
9101eab6178c Issue #15343: Handle importlib.machinery.FileFinder instances in pkgutil.walk_packages (et al) Nick Coghlan <ncoghlan@gmail.com> parents: 78107 diff changeset	160 iter_importer_modules.register(
9101eab6178c Issue #15343: Handle importlib.machinery.FileFinder instances in pkgutil.walk_packages (et al) Nick Coghlan <ncoghlan@gmail.com> parents: 78107 diff changeset	161 importlib.machinery.FileFinder, _iter_file_finder_modules)
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	162
84167 1b8f08c4efd5 Issue #17177: The imp module is pending deprecation. Brett Cannon <brett@python.org> parents: 84039 diff changeset	163
1b8f08c4efd5 Issue #17177: The imp module is pending deprecation. Brett Cannon <brett@python.org> parents: 84039 diff changeset	164 def _import_imp():
1b8f08c4efd5 Issue #17177: The imp module is pending deprecation. Brett Cannon <brett@python.org> parents: 84039 diff changeset	165 global imp
1b8f08c4efd5 Issue #17177: The imp module is pending deprecation. Brett Cannon <brett@python.org> parents: 84039 diff changeset	166 with warnings.catch_warnings():
1b8f08c4efd5 Issue #17177: The imp module is pending deprecation. Brett Cannon <brett@python.org> parents: 84039 diff changeset	167 warnings.simplefilter('ignore', PendingDeprecationWarning)
1b8f08c4efd5 Issue #17177: The imp module is pending deprecation. Brett Cannon <brett@python.org> parents: 84039 diff changeset	168 imp = importlib.import_module('imp')
1b8f08c4efd5 Issue #17177: The imp module is pending deprecation. Brett Cannon <brett@python.org> parents: 84039 diff changeset	169
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	170 class ImpImporter:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	171 """PEP 302 Importer that wraps Python's "classic" import algorithm
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	172
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	173 ImpImporter(dirname) produces a PEP 302 importer that searches that
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	174 directory. ImpImporter(None) produces a PEP 302 importer that searches
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	175 the current sys.path, plus any modules that are frozen or built-in.
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	176
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	177 Note that ImpImporter does not currently support being used by placement
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	178 on sys.meta_path.
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	179 """
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	180
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	181 def __init__(self, path=None):
84167 1b8f08c4efd5 Issue #17177: The imp module is pending deprecation. Brett Cannon <brett@python.org> parents: 84039 diff changeset	182 global imp
1b8f08c4efd5 Issue #17177: The imp module is pending deprecation. Brett Cannon <brett@python.org> parents: 84039 diff changeset	183 warnings.warn("This emulation is deprecated, use 'importlib' instead",
78107 3987667bf98f Take the first step in resolving the messy pkgutil vs importlib edge cases by basing pkgutil explicitly on importlib, deprecating its internal import emulation and setting __main__.__loader__ correctly so that runpy still works (Affects #15343, #15314, #15357) Nick Coghlan <ncoghlan@gmail.com> parents: 78038 diff changeset	184 DeprecationWarning)
84167 1b8f08c4efd5 Issue #17177: The imp module is pending deprecation. Brett Cannon <brett@python.org> parents: 84039 diff changeset	185 _import_imp()
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	186 self.path = path
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	187
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	188 def find_module(self, fullname, path=None):
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	189 # Note: we ignore 'path' argument since it is only used via meta_path
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	190 subname = fullname.split(".")[-1]
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	191 if subname != fullname and self.path is None:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	192 return None
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	193 if self.path is None:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	194 path = None
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	195 else:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	196 path = [os.path.realpath(self.path)]
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	197 try:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	198 file, filename, etc = imp.find_module(subname, path)
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	199 except ImportError:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	200 return None
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	201 return ImpLoader(fullname, file, filename, etc)
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	202
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	203 def iter_modules(self, prefix=''):
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	204 if self.path is None or not os.path.isdir(self.path):
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	205 return
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	206
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	207 yielded = {}
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	208 import inspect
72774 77bac85f610a Issue #7367: Fix pkgutil.walk_paths to skip directories whose Ned Deily <nad@acm.org> parents: 70410 diff changeset	209 try:
77bac85f610a Issue #7367: Fix pkgutil.walk_paths to skip directories whose Ned Deily <nad@acm.org> parents: 70410 diff changeset	210 filenames = os.listdir(self.path)
77bac85f610a Issue #7367: Fix pkgutil.walk_paths to skip directories whose Ned Deily <nad@acm.org> parents: 70410 diff changeset	211 except OSError:
77bac85f610a Issue #7367: Fix pkgutil.walk_paths to skip directories whose Ned Deily <nad@acm.org> parents: 70410 diff changeset	212 # ignore unreadable directories like import does
77bac85f610a Issue #7367: Fix pkgutil.walk_paths to skip directories whose Ned Deily <nad@acm.org> parents: 70410 diff changeset	213 filenames = []
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	214 filenames.sort() # handle packages before same-named modules
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	215
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	216 for fn in filenames:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	217 modname = inspect.getmodulename(fn)
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	218 if modname=='__init__' or modname in yielded:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	219 continue
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	220
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	221 path = os.path.join(self.path, fn)
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	222 ispkg = False
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	223
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	224 if not modname and os.path.isdir(path) and '.' not in fn:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	225 modname = fn
72774 77bac85f610a Issue #7367: Fix pkgutil.walk_paths to skip directories whose Ned Deily <nad@acm.org> parents: 70410 diff changeset	226 try:
77bac85f610a Issue #7367: Fix pkgutil.walk_paths to skip directories whose Ned Deily <nad@acm.org> parents: 70410 diff changeset	227 dircontents = os.listdir(path)
77bac85f610a Issue #7367: Fix pkgutil.walk_paths to skip directories whose Ned Deily <nad@acm.org> parents: 70410 diff changeset	228 except OSError:
77bac85f610a Issue #7367: Fix pkgutil.walk_paths to skip directories whose Ned Deily <nad@acm.org> parents: 70410 diff changeset	229 # ignore unreadable directories like import does
77bac85f610a Issue #7367: Fix pkgutil.walk_paths to skip directories whose Ned Deily <nad@acm.org> parents: 70410 diff changeset	230 dircontents = []
77bac85f610a Issue #7367: Fix pkgutil.walk_paths to skip directories whose Ned Deily <nad@acm.org> parents: 70410 diff changeset	231 for fn in dircontents:
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	232 subname = inspect.getmodulename(fn)
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	233 if subname=='__init__':
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	234 ispkg = True
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	235 break
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	236 else:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	237 continue # not a package
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	238
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	239 if modname and '.' not in modname:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	240 yielded[modname] = 1
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	241 yield prefix + modname, ispkg
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	242
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	243
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	244 class ImpLoader:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	245 """PEP 302 Loader that wraps Python's "classic" import algorithm
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	246 """
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	247 code = source = None
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	248
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	249 def __init__(self, fullname, file, filename, etc):
84167 1b8f08c4efd5 Issue #17177: The imp module is pending deprecation. Brett Cannon <brett@python.org> parents: 84039 diff changeset	250 warnings.warn("This emulation is deprecated, use 'importlib' instead",
1b8f08c4efd5 Issue #17177: The imp module is pending deprecation. Brett Cannon <brett@python.org> parents: 84039 diff changeset	251 DeprecationWarning)
1b8f08c4efd5 Issue #17177: The imp module is pending deprecation. Brett Cannon <brett@python.org> parents: 84039 diff changeset	252 _import_imp()
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	253 self.file = file
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	254 self.filename = filename
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	255 self.fullname = fullname
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	256 self.etc = etc
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	257
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	258 def load_module(self, fullname):
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	259 self._reopen()
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	260 try:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	261 mod = imp.load_module(fullname, self.file, self.filename, self.etc)
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	262 finally:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	263 if self.file:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	264 self.file.close()
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	265 # Note: we don't set __loader__ because we want the module to look
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	266 # normal; i.e. this is just a wrapper for standard import machinery
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	267 return mod
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	268
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	269 def get_data(self, pathname):
65730 d161714e7c92 Have pkgutil properly close files. Brett Cannon <bcannon@gmail.com> parents: 55602 diff changeset	270 with open(pathname, "rb") as file:
d161714e7c92 Have pkgutil properly close files. Brett Cannon <bcannon@gmail.com> parents: 55602 diff changeset	271 return file.read()
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	272
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	273 def _reopen(self):
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	274 if self.file and self.file.closed:
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	275 mod_type = self.etc[2]
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	276 if mod_type==imp.PY_SOURCE:
69823 be5b8d1ded34 Replace open(filename, 'rU') by open(filename, 'r') Victor Stinner <victor.stinner@haypocalc.com> parents: 65730 diff changeset	277 self.file = open(self.filename, 'r')
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	278 elif mod_type in (imp.PY_COMPILED, imp.C_EXTENSION):
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	279 self.file = open(self.filename, 'rb')
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	280
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	281 def _fix_name(self, fullname):
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	282 if fullname is None:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	283 fullname = self.fullname
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	284 elif fullname != self.fullname:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	285 raise ImportError("Loader for module %s cannot handle "
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	286 "module %s" % (self.fullname, fullname))
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	287 return fullname
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	288
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	289 def is_package(self, fullname):
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	290 fullname = self._fix_name(fullname)
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	291 return self.etc[2]==imp.PKG_DIRECTORY
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	292
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	293 def get_code(self, fullname=None):
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	294 fullname = self._fix_name(fullname)
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	295 if self.code is None:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	296 mod_type = self.etc[2]
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	297 if mod_type==imp.PY_SOURCE:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	298 source = self.get_source(fullname)
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	299 self.code = compile(source, self.filename, 'exec')
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	300 elif mod_type==imp.PY_COMPILED:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	301 self._reopen()
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	302 try:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	303 self.code = read_code(self.file)
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	304 finally:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	305 self.file.close()
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	306 elif mod_type==imp.PKG_DIRECTORY:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	307 self.code = self._get_delegate().get_code()
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	308 return self.code
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	309
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	310 def get_source(self, fullname=None):
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	311 fullname = self._fix_name(fullname)
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	312 if self.source is None:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	313 mod_type = self.etc[2]
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	314 if mod_type==imp.PY_SOURCE:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	315 self._reopen()
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	316 try:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	317 self.source = self.file.read()
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	318 finally:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	319 self.file.close()
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	320 elif mod_type==imp.PY_COMPILED:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	321 if os.path.exists(self.filename[:-1]):
82168 cb876235f29d modernize some modules' code by using with statement around open() Giampaolo Rodola' <g.rodola@gmail.com> parents: 81039 diff changeset	322 with open(self.filename[:-1], 'r') as f:
cb876235f29d modernize some modules' code by using with statement around open() Giampaolo Rodola' <g.rodola@gmail.com> parents: 81039 diff changeset	323 self.source = f.read()
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	324 elif mod_type==imp.PKG_DIRECTORY:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	325 self.source = self._get_delegate().get_source()
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	326 return self.source
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	327
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	328
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	329 def _get_delegate(self):
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	330 return ImpImporter(self.filename).find_module('__init__')
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	331
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	332 def get_filename(self, fullname=None):
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	333 fullname = self._fix_name(fullname)
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	334 mod_type = self.etc[2]
72402 f18e9db83483 Use a local name (it was intended to be used but overlooked). Éric Araujo <merwok@netwok.org> parents: 70413 diff changeset	335 if mod_type==imp.PKG_DIRECTORY:
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	336 return self._get_delegate().get_filename()
72402 f18e9db83483 Use a local name (it was intended to be used but overlooked). Éric Araujo <merwok@netwok.org> parents: 70413 diff changeset	337 elif mod_type in (imp.PY_SOURCE, imp.PY_COMPILED, imp.C_EXTENSION):
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	338 return self.filename
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	339 return None
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	340
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	341
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	342 try:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	343 import zipimport
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	344 from zipimport import zipimporter
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	345
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	346 def iter_zipimport_modules(importer, prefix=''):
55602 680ad8eed33b Merged revisions 73833,73838,73850-73852,73856-73857 via svnmerge from Georg Brandl <georg@python.org> parents: 51657 diff changeset	347 dirlist = sorted(zipimport._zip_directory_cache[importer.archive])
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	348 _prefix = importer.prefix
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	349 plen = len(_prefix)
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	350 yielded = {}
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	351 import inspect
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	352 for fn in dirlist:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	353 if not fn.startswith(_prefix):
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	354 continue
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	355
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	356 fn = fn[plen:].split(os.sep)
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	357
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	358 if len(fn)==2 and fn[1].startswith('__init__.py'):
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	359 if fn[0] not in yielded:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	360 yielded[fn[0]] = 1
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	361 yield fn[0], True
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	362
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	363 if len(fn)!=1:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	364 continue
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	365
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	366 modname = inspect.getmodulename(fn[0])
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	367 if modname=='__init__':
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	368 continue
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	369
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	370 if modname and '.' not in modname and modname not in yielded:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	371 yielded[modname] = 1
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	372 yield prefix + modname, False
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	373
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	374 iter_importer_modules.register(zipimporter, iter_zipimport_modules)
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	375
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	376 except ImportError:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	377 pass
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	378
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	379
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	380 def get_importer(path_item):
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	381 """Retrieve a PEP 302 importer for the given path item
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	382
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	383 The returned importer is cached in sys.path_importer_cache
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	384 if it was newly created by a path hook.
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	385
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	386 The cache (or part of it) can be cleared manually if a
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	387 rescan of sys.path_hooks is necessary.
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	388 """
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	389 try:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	390 importer = sys.path_importer_cache[path_item]
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	391 except KeyError:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	392 for path_hook in sys.path_hooks:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	393 try:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	394 importer = path_hook(path_item)
76566 57d558f1904d Issue #14605: Make explicit the entries on sys.path_hooks that used to Brett Cannon <brett@python.org> parents: 74358 diff changeset	395 sys.path_importer_cache.setdefault(path_item, importer)
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	396 break
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	397 except ImportError:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	398 pass
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	399 else:
78107 3987667bf98f Take the first step in resolving the messy pkgutil vs importlib edge cases by basing pkgutil explicitly on importlib, deprecating its internal import emulation and setting __main__.__loader__ correctly so that runpy still works (Affects #15343, #15314, #15357) Nick Coghlan <ncoghlan@gmail.com> parents: 78038 diff changeset	400 importer = None
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	401 return importer
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	402
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	403
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	404 def iter_importers(fullname=""):
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	405 """Yield PEP 302 importers for the given module name
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	406
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	407 If fullname contains a '.', the importers will be for the package
78107 3987667bf98f Take the first step in resolving the messy pkgutil vs importlib edge cases by basing pkgutil explicitly on importlib, deprecating its internal import emulation and setting __main__.__loader__ correctly so that runpy still works (Affects #15343, #15314, #15357) Nick Coghlan <ncoghlan@gmail.com> parents: 78038 diff changeset	408 containing fullname, otherwise they will be all registered top level
3987667bf98f Take the first step in resolving the messy pkgutil vs importlib edge cases by basing pkgutil explicitly on importlib, deprecating its internal import emulation and setting __main__.__loader__ correctly so that runpy still works (Affects #15343, #15314, #15357) Nick Coghlan <ncoghlan@gmail.com> parents: 78038 diff changeset	409 importers (i.e. those on both sys.meta_path and sys.path_hooks).
3987667bf98f Take the first step in resolving the messy pkgutil vs importlib edge cases by basing pkgutil explicitly on importlib, deprecating its internal import emulation and setting __main__.__loader__ correctly so that runpy still works (Affects #15343, #15314, #15357) Nick Coghlan <ncoghlan@gmail.com> parents: 78038 diff changeset	410
3987667bf98f Take the first step in resolving the messy pkgutil vs importlib edge cases by basing pkgutil explicitly on importlib, deprecating its internal import emulation and setting __main__.__loader__ correctly so that runpy still works (Affects #15343, #15314, #15357) Nick Coghlan <ncoghlan@gmail.com> parents: 78038 diff changeset	411 If the named module is in a package, that package is imported as a side
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	412 effect of invoking this function.
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	413
78107 3987667bf98f Take the first step in resolving the messy pkgutil vs importlib edge cases by basing pkgutil explicitly on importlib, deprecating its internal import emulation and setting __main__.__loader__ correctly so that runpy still works (Affects #15343, #15314, #15357) Nick Coghlan <ncoghlan@gmail.com> parents: 78038 diff changeset	414 If no module name is specified, all top level importers are produced.
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	415 """
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	416 if fullname.startswith('.'):
78107 3987667bf98f Take the first step in resolving the messy pkgutil vs importlib edge cases by basing pkgutil explicitly on importlib, deprecating its internal import emulation and setting __main__.__loader__ correctly so that runpy still works (Affects #15343, #15314, #15357) Nick Coghlan <ncoghlan@gmail.com> parents: 78038 diff changeset	417 msg = "Relative module name {!r} not supported".format(fullname)
3987667bf98f Take the first step in resolving the messy pkgutil vs importlib edge cases by basing pkgutil explicitly on importlib, deprecating its internal import emulation and setting __main__.__loader__ correctly so that runpy still works (Affects #15343, #15314, #15357) Nick Coghlan <ncoghlan@gmail.com> parents: 78038 diff changeset	418 raise ImportError(msg)
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	419 if '.' in fullname:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	420 # Get the containing package's __path__
78107 3987667bf98f Take the first step in resolving the messy pkgutil vs importlib edge cases by basing pkgutil explicitly on importlib, deprecating its internal import emulation and setting __main__.__loader__ correctly so that runpy still works (Affects #15343, #15314, #15357) Nick Coghlan <ncoghlan@gmail.com> parents: 78038 diff changeset	421 pkg_name = fullname.rpartition(".")[0]
83366 c40b50d65a00 Close issue #16163: handle submodules in pkgutil.iter_importers Nick Coghlan <ncoghlan@gmail.com> parents: 80950 diff changeset	422 pkg = importlib.import_module(pkg_name)
c40b50d65a00 Close issue #16163: handle submodules in pkgutil.iter_importers Nick Coghlan <ncoghlan@gmail.com> parents: 80950 diff changeset	423 path = getattr(pkg, '__path__', None)
78107 3987667bf98f Take the first step in resolving the messy pkgutil vs importlib edge cases by basing pkgutil explicitly on importlib, deprecating its internal import emulation and setting __main__.__loader__ correctly so that runpy still works (Affects #15343, #15314, #15357) Nick Coghlan <ncoghlan@gmail.com> parents: 78038 diff changeset	424 if path is None:
3987667bf98f Take the first step in resolving the messy pkgutil vs importlib edge cases by basing pkgutil explicitly on importlib, deprecating its internal import emulation and setting __main__.__loader__ correctly so that runpy still works (Affects #15343, #15314, #15357) Nick Coghlan <ncoghlan@gmail.com> parents: 78038 diff changeset	425 return
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	426 else:
79583 70d49694786c Issue #16120: Use \|yield from\| in stdlib. Andrew Svetlov <andrew.svetlov@gmail.com> parents: 79378 diff changeset	427 yield from sys.meta_path
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	428 path = sys.path
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	429 for item in path:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	430 yield get_importer(item)
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	431
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	432 def get_loader(module_or_name):
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	433 """Get a PEP 302 "loader" object for module_or_name
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	434
78107 3987667bf98f Take the first step in resolving the messy pkgutil vs importlib edge cases by basing pkgutil explicitly on importlib, deprecating its internal import emulation and setting __main__.__loader__ correctly so that runpy still works (Affects #15343, #15314, #15357) Nick Coghlan <ncoghlan@gmail.com> parents: 78038 diff changeset	435 Returns None if the module cannot be found or imported.
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	436 If the named module is not already imported, its containing package
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	437 (if any) is imported, in order to establish the package __path__.
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	438 """
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	439 if module_or_name in sys.modules:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	440 module_or_name = sys.modules[module_or_name]
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	441 if isinstance(module_or_name, ModuleType):
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	442 module = module_or_name
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 37879 diff changeset	443 loader = getattr(module, '__loader__', None)
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	444 if loader is not None:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	445 return loader
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	446 fullname = module.__name__
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	447 else:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	448 fullname = module_or_name
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	449 return find_loader(fullname)
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	450
78107 3987667bf98f Take the first step in resolving the messy pkgutil vs importlib edge cases by basing pkgutil explicitly on importlib, deprecating its internal import emulation and setting __main__.__loader__ correctly so that runpy still works (Affects #15343, #15314, #15357) Nick Coghlan <ncoghlan@gmail.com> parents: 78038 diff changeset	451
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	452 def find_loader(fullname):
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	453 """Find a PEP 302 "loader" object for fullname
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	454
78107 3987667bf98f Take the first step in resolving the messy pkgutil vs importlib edge cases by basing pkgutil explicitly on importlib, deprecating its internal import emulation and setting __main__.__loader__ correctly so that runpy still works (Affects #15343, #15314, #15357) Nick Coghlan <ncoghlan@gmail.com> parents: 78038 diff changeset	455 This is s convenience wrapper around :func:`importlib.find_loader` that
3987667bf98f Take the first step in resolving the messy pkgutil vs importlib edge cases by basing pkgutil explicitly on importlib, deprecating its internal import emulation and setting __main__.__loader__ correctly so that runpy still works (Affects #15343, #15314, #15357) Nick Coghlan <ncoghlan@gmail.com> parents: 78038 diff changeset	456 sets the path argument correctly when searching for submodules, and
3987667bf98f Take the first step in resolving the messy pkgutil vs importlib edge cases by basing pkgutil explicitly on importlib, deprecating its internal import emulation and setting __main__.__loader__ correctly so that runpy still works (Affects #15343, #15314, #15357) Nick Coghlan <ncoghlan@gmail.com> parents: 78038 diff changeset	457 also ensures parent packages (if any) are imported before searching for
3987667bf98f Take the first step in resolving the messy pkgutil vs importlib edge cases by basing pkgutil explicitly on importlib, deprecating its internal import emulation and setting __main__.__loader__ correctly so that runpy still works (Affects #15343, #15314, #15357) Nick Coghlan <ncoghlan@gmail.com> parents: 78038 diff changeset	458 submodules.
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	459 """
78107 3987667bf98f Take the first step in resolving the messy pkgutil vs importlib edge cases by basing pkgutil explicitly on importlib, deprecating its internal import emulation and setting __main__.__loader__ correctly so that runpy still works (Affects #15343, #15314, #15357) Nick Coghlan <ncoghlan@gmail.com> parents: 78038 diff changeset	460 if fullname.startswith('.'):
3987667bf98f Take the first step in resolving the messy pkgutil vs importlib edge cases by basing pkgutil explicitly on importlib, deprecating its internal import emulation and setting __main__.__loader__ correctly so that runpy still works (Affects #15343, #15314, #15357) Nick Coghlan <ncoghlan@gmail.com> parents: 78038 diff changeset	461 msg = "Relative module name {!r} not supported".format(fullname)
3987667bf98f Take the first step in resolving the messy pkgutil vs importlib edge cases by basing pkgutil explicitly on importlib, deprecating its internal import emulation and setting __main__.__loader__ correctly so that runpy still works (Affects #15343, #15314, #15357) Nick Coghlan <ncoghlan@gmail.com> parents: 78038 diff changeset	462 raise ImportError(msg)
3987667bf98f Take the first step in resolving the messy pkgutil vs importlib edge cases by basing pkgutil explicitly on importlib, deprecating its internal import emulation and setting __main__.__loader__ correctly so that runpy still works (Affects #15343, #15314, #15357) Nick Coghlan <ncoghlan@gmail.com> parents: 78038 diff changeset	463 path = None
3987667bf98f Take the first step in resolving the messy pkgutil vs importlib edge cases by basing pkgutil explicitly on importlib, deprecating its internal import emulation and setting __main__.__loader__ correctly so that runpy still works (Affects #15343, #15314, #15357) Nick Coghlan <ncoghlan@gmail.com> parents: 78038 diff changeset	464 pkg_name = fullname.rpartition(".")[0]
3987667bf98f Take the first step in resolving the messy pkgutil vs importlib edge cases by basing pkgutil explicitly on importlib, deprecating its internal import emulation and setting __main__.__loader__ correctly so that runpy still works (Affects #15343, #15314, #15357) Nick Coghlan <ncoghlan@gmail.com> parents: 78038 diff changeset	465 if pkg_name:
3987667bf98f Take the first step in resolving the messy pkgutil vs importlib edge cases by basing pkgutil explicitly on importlib, deprecating its internal import emulation and setting __main__.__loader__ correctly so that runpy still works (Affects #15343, #15314, #15357) Nick Coghlan <ncoghlan@gmail.com> parents: 78038 diff changeset	466 pkg = importlib.import_module(pkg_name)
3987667bf98f Take the first step in resolving the messy pkgutil vs importlib edge cases by basing pkgutil explicitly on importlib, deprecating its internal import emulation and setting __main__.__loader__ correctly so that runpy still works (Affects #15343, #15314, #15357) Nick Coghlan <ncoghlan@gmail.com> parents: 78038 diff changeset	467 path = getattr(pkg, "__path__", None)
3987667bf98f Take the first step in resolving the messy pkgutil vs importlib edge cases by basing pkgutil explicitly on importlib, deprecating its internal import emulation and setting __main__.__loader__ correctly so that runpy still works (Affects #15343, #15314, #15357) Nick Coghlan <ncoghlan@gmail.com> parents: 78038 diff changeset	468 if path is None:
3987667bf98f Take the first step in resolving the messy pkgutil vs importlib edge cases by basing pkgutil explicitly on importlib, deprecating its internal import emulation and setting __main__.__loader__ correctly so that runpy still works (Affects #15343, #15314, #15357) Nick Coghlan <ncoghlan@gmail.com> parents: 78038 diff changeset	469 return None
3987667bf98f Take the first step in resolving the messy pkgutil vs importlib edge cases by basing pkgutil explicitly on importlib, deprecating its internal import emulation and setting __main__.__loader__ correctly so that runpy still works (Affects #15343, #15314, #15357) Nick Coghlan <ncoghlan@gmail.com> parents: 78038 diff changeset	470 try:
3987667bf98f Take the first step in resolving the messy pkgutil vs importlib edge cases by basing pkgutil explicitly on importlib, deprecating its internal import emulation and setting __main__.__loader__ correctly so that runpy still works (Affects #15343, #15314, #15357) Nick Coghlan <ncoghlan@gmail.com> parents: 78038 diff changeset	471 return importlib.find_loader(fullname, path)
3987667bf98f Take the first step in resolving the messy pkgutil vs importlib edge cases by basing pkgutil explicitly on importlib, deprecating its internal import emulation and setting __main__.__loader__ correctly so that runpy still works (Affects #15343, #15314, #15357) Nick Coghlan <ncoghlan@gmail.com> parents: 78038 diff changeset	472 except (ImportError, AttributeError, TypeError, ValueError) as ex:
3987667bf98f Take the first step in resolving the messy pkgutil vs importlib edge cases by basing pkgutil explicitly on importlib, deprecating its internal import emulation and setting __main__.__loader__ correctly so that runpy still works (Affects #15343, #15314, #15357) Nick Coghlan <ncoghlan@gmail.com> parents: 78038 diff changeset	473 # This hack fixes an impedance mismatch between pkgutil and
80929 15a391919deb Issue #16714: use 'raise' exceptions, don't 'throw'. Andrew Svetlov <andrew.svetlov@gmail.com> parents: 78195 diff changeset	474 # importlib, where the latter raises other errors for cases where
80950 55d86476d048 replace threw with raised (#16714) Andrew Svetlov <andrew.svetlov@gmail.com> parents: 80929 diff changeset	475 # pkgutil previously raised ImportError
78107 3987667bf98f Take the first step in resolving the messy pkgutil vs importlib edge cases by basing pkgutil explicitly on importlib, deprecating its internal import emulation and setting __main__.__loader__ correctly so that runpy still works (Affects #15343, #15314, #15357) Nick Coghlan <ncoghlan@gmail.com> parents: 78038 diff changeset	476 msg = "Error while finding loader for {!r} ({}: {})"
3987667bf98f Take the first step in resolving the messy pkgutil vs importlib edge cases by basing pkgutil explicitly on importlib, deprecating its internal import emulation and setting __main__.__loader__ correctly so that runpy still works (Affects #15343, #15314, #15357) Nick Coghlan <ncoghlan@gmail.com> parents: 78038 diff changeset	477 raise ImportError(msg.format(fullname, type(ex), ex)) from ex
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 30448 diff changeset	478
26753 7c15bf7726b2 Last week we discussed adding this module to the standard library. Guido van Rossum <guido@python.org> parents: diff changeset	479
7c15bf7726b2 Last week we discussed adding this module to the standard library. Guido van Rossum <guido@python.org> parents: diff changeset	480 def extend_path(path, name):
7c15bf7726b2 Last week we discussed adding this module to the standard library. Guido van Rossum <guido@python.org> parents: diff changeset	481 """Extend a package's path.
7c15bf7726b2 Last week we discussed adding this module to the standard library. Guido van Rossum <guido@python.org> parents: diff changeset	482
7c15bf7726b2 Last week we discussed adding this module to the standard library. Guido van Rossum <guido@python.org> parents: diff changeset	483 Intended use is to place the following code in a package's __init__.py:
7c15bf7726b2 Last week we discussed adding this module to the standard library. Guido van Rossum <guido@python.org> parents: diff changeset	484
7c15bf7726b2 Last week we discussed adding this module to the standard library. Guido van Rossum <guido@python.org> parents: diff changeset	485 from pkgutil import extend_path
7c15bf7726b2 Last week we discussed adding this module to the standard library. Guido van Rossum <guido@python.org> parents: diff changeset	486 __path__ = extend_path(__path__, __name__)
7c15bf7726b2 Last week we discussed adding this module to the standard library. Guido van Rossum <guido@python.org> parents: diff changeset	487
7c15bf7726b2 Last week we discussed adding this module to the standard library. Guido van Rossum <guido@python.org> parents: diff changeset	488 This will add to the package's __path__ all subdirectories of
7c15bf7726b2 Last week we discussed adding this module to the standard library. Guido van Rossum <guido@python.org> parents: diff changeset	489 directories on sys.path named after the package. This is useful
7c15bf7726b2 Last week we discussed adding this module to the standard library. Guido van Rossum <guido@python.org> parents: diff changeset	490 if one wants to distribute different parts of a single logical
7c15bf7726b2 Last week we discussed adding this module to the standard library. Guido van Rossum <guido@python.org> parents: diff changeset	491 package as multiple directories.
7c15bf7726b2 Last week we discussed adding this module to the standard library. Guido van Rossum <guido@python.org> parents: diff changeset	492
7c15bf7726b2 Last week we discussed adding this module to the standard library. Guido van Rossum <guido@python.org> parents: diff changeset	493 It also looks for .pkg files beginning where matches the name
7c15bf7726b2 Last week we discussed adding this module to the standard library. Guido van Rossum <guido@python.org> parents: diff changeset	494 argument. This feature is similar to *.pth files (see site.py),
7c15bf7726b2 Last week we discussed adding this module to the standard library. Guido van Rossum <guido@python.org> parents: diff changeset	495 except that it doesn't special-case lines starting with 'import'.
7c15bf7726b2 Last week we discussed adding this module to the standard library. Guido van Rossum <guido@python.org> parents: diff changeset	496 A *.pkg file is trusted at face value: apart from checking for
7c15bf7726b2 Last week we discussed adding this module to the standard library. Guido van Rossum <guido@python.org> parents: diff changeset	497 duplicates, all entries found in a *.pkg file are added to the
7c15bf7726b2 Last week we discussed adding this module to the standard library. Guido van Rossum <guido@python.org> parents: diff changeset	498 path, regardless of whether they are exist the filesystem. (This
7c15bf7726b2 Last week we discussed adding this module to the standard library. Guido van Rossum <guido@python.org> parents: diff changeset	499 is a feature.)
7c15bf7726b2 Last week we discussed adding this module to the standard library. Guido van Rossum <guido@python.org> parents: diff changeset	500
7c15bf7726b2 Last week we discussed adding this module to the standard library. Guido van Rossum <guido@python.org> parents: diff changeset	501 If the input path is not a list (as is the case for frozen
7c15bf7726b2 Last week we discussed adding this module to the standard library. Guido van Rossum <guido@python.org> parents: diff changeset	502 packages) it is returned unchanged. The input path is not
7c15bf7726b2 Last week we discussed adding this module to the standard library. Guido van Rossum <guido@python.org> parents: diff changeset	503 modified; an extended copy is returned. Items are only appended
7c15bf7726b2 Last week we discussed adding this module to the standard library. Guido van Rossum <guido@python.org> parents: diff changeset	504 to the copy at the end.
7c15bf7726b2 Last week we discussed adding this module to the standard library. Guido van Rossum <guido@python.org> parents: diff changeset	505
7c15bf7726b2 Last week we discussed adding this module to the standard library. Guido van Rossum <guido@python.org> parents: diff changeset	506 It is assumed that sys.path is a sequence. Items of sys.path that
7c15bf7726b2 Last week we discussed adding this module to the standard library. Guido van Rossum <guido@python.org> parents: diff changeset	507 are not (unicode or 8-bit) strings referring to existing
7c15bf7726b2 Last week we discussed adding this module to the standard library. Guido van Rossum <guido@python.org> parents: diff changeset	508 directories are ignored. Unicode items of sys.path that cause
7c15bf7726b2 Last week we discussed adding this module to the standard library. Guido van Rossum <guido@python.org> parents: diff changeset	509 errors when used as filenames may cause this function to raise an
7c15bf7726b2 Last week we discussed adding this module to the standard library. Guido van Rossum <guido@python.org> parents: diff changeset	510 exception (in line with os.path.isdir() behavior).
7c15bf7726b2 Last week we discussed adding this module to the standard library. Guido van Rossum <guido@python.org> parents: diff changeset	511 """
7c15bf7726b2 Last week we discussed adding this module to the standard library. Guido van Rossum <guido@python.org> parents: diff changeset	512
7c15bf7726b2 Last week we discussed adding this module to the standard library. Guido van Rossum <guido@python.org> parents: diff changeset	513 if not isinstance(path, list):
7c15bf7726b2 Last week we discussed adding this module to the standard library. Guido van Rossum <guido@python.org> parents: diff changeset	514 # This could happen e.g. when this is called from inside a
7c15bf7726b2 Last week we discussed adding this module to the standard library. Guido van Rossum <guido@python.org> parents: diff changeset	515 # frozen package. Return the path unchanged in that case.
7c15bf7726b2 Last week we discussed adding this module to the standard library. Guido van Rossum <guido@python.org> parents: diff changeset	516 return path
7c15bf7726b2 Last week we discussed adding this module to the standard library. Guido van Rossum <guido@python.org> parents: diff changeset	517
42622 4d2aa4bfb2d6 Remove RISCOS support Skip Montanaro <skip@pobox.com> parents: 40587 diff changeset	518 sname_pkg = name + ".pkg"
26753 7c15bf7726b2 Last week we discussed adding this module to the standard library. Guido van Rossum <guido@python.org> parents: diff changeset	519
7c15bf7726b2 Last week we discussed adding this module to the standard library. Guido van Rossum <guido@python.org> parents: diff changeset	520 path = path[:] # Start with a copy of the existing path
7c15bf7726b2 Last week we discussed adding this module to the standard library. Guido van Rossum <guido@python.org> parents: diff changeset	521
78038 a7b8c3323db9 Issue #15294: Fix a regression in pkgutil.extend_path()'s handling of nested namespace packages. Antoine Pitrou <solipsis@pitrou.net> parents: 77126 diff changeset	522 parent_package, _, final_name = name.rpartition('.')
a7b8c3323db9 Issue #15294: Fix a regression in pkgutil.extend_path()'s handling of nested namespace packages. Antoine Pitrou <solipsis@pitrou.net> parents: 77126 diff changeset	523 if parent_package:
a7b8c3323db9 Issue #15294: Fix a regression in pkgutil.extend_path()'s handling of nested namespace packages. Antoine Pitrou <solipsis@pitrou.net> parents: 77126 diff changeset	524 try:
a7b8c3323db9 Issue #15294: Fix a regression in pkgutil.extend_path()'s handling of nested namespace packages. Antoine Pitrou <solipsis@pitrou.net> parents: 77126 diff changeset	525 search_path = sys.modules[parent_package].__path__
a7b8c3323db9 Issue #15294: Fix a regression in pkgutil.extend_path()'s handling of nested namespace packages. Antoine Pitrou <solipsis@pitrou.net> parents: 77126 diff changeset	526 except (KeyError, AttributeError):
a7b8c3323db9 Issue #15294: Fix a regression in pkgutil.extend_path()'s handling of nested namespace packages. Antoine Pitrou <solipsis@pitrou.net> parents: 77126 diff changeset	527 # We can't do anything: find_loader() returns None when
a7b8c3323db9 Issue #15294: Fix a regression in pkgutil.extend_path()'s handling of nested namespace packages. Antoine Pitrou <solipsis@pitrou.net> parents: 77126 diff changeset	528 # passed a dotted name.
a7b8c3323db9 Issue #15294: Fix a regression in pkgutil.extend_path()'s handling of nested namespace packages. Antoine Pitrou <solipsis@pitrou.net> parents: 77126 diff changeset	529 return path
a7b8c3323db9 Issue #15294: Fix a regression in pkgutil.extend_path()'s handling of nested namespace packages. Antoine Pitrou <solipsis@pitrou.net> parents: 77126 diff changeset	530 else:
a7b8c3323db9 Issue #15294: Fix a regression in pkgutil.extend_path()'s handling of nested namespace packages. Antoine Pitrou <solipsis@pitrou.net> parents: 77126 diff changeset	531 search_path = sys.path
a7b8c3323db9 Issue #15294: Fix a regression in pkgutil.extend_path()'s handling of nested namespace packages. Antoine Pitrou <solipsis@pitrou.net> parents: 77126 diff changeset	532
a7b8c3323db9 Issue #15294: Fix a regression in pkgutil.extend_path()'s handling of nested namespace packages. Antoine Pitrou <solipsis@pitrou.net> parents: 77126 diff changeset	533 for dir in search_path:
77126 702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 76584 diff changeset	534 if not isinstance(dir, str):
26753 7c15bf7726b2 Last week we discussed adding this module to the standard library. Guido van Rossum <guido@python.org> parents: diff changeset	535 continue
77126 702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 76584 diff changeset	536
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 76584 diff changeset	537 finder = get_importer(dir)
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 76584 diff changeset	538 if finder is not None:
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 76584 diff changeset	539 # Is this finder PEP 420 compliant?
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 76584 diff changeset	540 if hasattr(finder, 'find_loader'):
78038 a7b8c3323db9 Issue #15294: Fix a regression in pkgutil.extend_path()'s handling of nested namespace packages. Antoine Pitrou <solipsis@pitrou.net> parents: 77126 diff changeset	541 loader, portions = finder.find_loader(final_name)
77126 702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 76584 diff changeset	542 else:
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 76584 diff changeset	543 # No, no need to call it
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 76584 diff changeset	544 loader = None
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 76584 diff changeset	545 portions = []
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 76584 diff changeset	546
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 76584 diff changeset	547 for portion in portions:
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 76584 diff changeset	548 # XXX This may still add duplicate entries to path on
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 76584 diff changeset	549 # case-insensitive filesystems
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 76584 diff changeset	550 if portion not in path:
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 76584 diff changeset	551 path.append(portion)
702009f3c0b1 issue 14660: Implement PEP 420, namespace packages. Eric V. Smith <eric@trueblade.com> parents: 76584 diff changeset	552
26753 7c15bf7726b2 Last week we discussed adding this module to the standard library. Guido van Rossum <guido@python.org> parents: diff changeset	553 # XXX Is this the right thing for subpackages like zope.app?
7c15bf7726b2 Last week we discussed adding this module to the standard library. Guido van Rossum <guido@python.org> parents: diff changeset	554 # It looks for a file named "zope.app.pkg"
7c15bf7726b2 Last week we discussed adding this module to the standard library. Guido van Rossum <guido@python.org> parents: diff changeset	555 pkgfile = os.path.join(dir, sname_pkg)
7c15bf7726b2 Last week we discussed adding this module to the standard library. Guido van Rossum <guido@python.org> parents: diff changeset	556 if os.path.isfile(pkgfile):
7c15bf7726b2 Last week we discussed adding this module to the standard library. Guido van Rossum <guido@python.org> parents: diff changeset	557 try:
7c15bf7726b2 Last week we discussed adding this module to the standard library. Guido van Rossum <guido@python.org> parents: diff changeset	558 f = open(pkgfile)
81039 7d69d04522e3 Replace IOError with OSError (#16715) Andrew Svetlov <andrew.svetlov@gmail.com> parents: 80951 diff changeset	559 except OSError as msg:
26753 7c15bf7726b2 Last week we discussed adding this module to the standard library. Guido van Rossum <guido@python.org> parents: diff changeset	560 sys.stderr.write("Can't open %s: %s\n" %
7c15bf7726b2 Last week we discussed adding this module to the standard library. Guido van Rossum <guido@python.org> parents: diff changeset	561 (pkgfile, msg))
7c15bf7726b2 Last week we discussed adding this module to the standard library. Guido van Rossum <guido@python.org> parents: diff changeset	562 else:
82168 cb876235f29d modernize some modules' code by using with statement around open() Giampaolo Rodola' <g.rodola@gmail.com> parents: 81039 diff changeset	563 with f:
cb876235f29d modernize some modules' code by using with statement around open() Giampaolo Rodola' <g.rodola@gmail.com> parents: 81039 diff changeset	564 for line in f:
cb876235f29d modernize some modules' code by using with statement around open() Giampaolo Rodola' <g.rodola@gmail.com> parents: 81039 diff changeset	565 line = line.rstrip('\n')
cb876235f29d modernize some modules' code by using with statement around open() Giampaolo Rodola' <g.rodola@gmail.com> parents: 81039 diff changeset	566 if not line or line.startswith('#'):
cb876235f29d modernize some modules' code by using with statement around open() Giampaolo Rodola' <g.rodola@gmail.com> parents: 81039 diff changeset	567 continue
cb876235f29d modernize some modules' code by using with statement around open() Giampaolo Rodola' <g.rodola@gmail.com> parents: 81039 diff changeset	568 path.append(line) # Don't check for existence!
26753 7c15bf7726b2 Last week we discussed adding this module to the standard library. Guido van Rossum <guido@python.org> parents: diff changeset	569
7c15bf7726b2 Last week we discussed adding this module to the standard library. Guido van Rossum <guido@python.org> parents: diff changeset	570 return path
46478 beb7abcd6c01 Merged revisions 62350-62355,62358-62359,62364-62365,62370,62372-62375,62378-62379,62381 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 43577 diff changeset	571
beb7abcd6c01 Merged revisions 62350-62355,62358-62359,62364-62365,62370,62372-62375,62378-62379,62381 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 43577 diff changeset	572 def get_data(package, resource):
beb7abcd6c01 Merged revisions 62350-62355,62358-62359,62364-62365,62370,62372-62375,62378-62379,62381 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 43577 diff changeset	573 """Get a resource from a package.
beb7abcd6c01 Merged revisions 62350-62355,62358-62359,62364-62365,62370,62372-62375,62378-62379,62381 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 43577 diff changeset	574
beb7abcd6c01 Merged revisions 62350-62355,62358-62359,62364-62365,62370,62372-62375,62378-62379,62381 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 43577 diff changeset	575 This is a wrapper round the PEP 302 loader get_data API. The package
beb7abcd6c01 Merged revisions 62350-62355,62358-62359,62364-62365,62370,62372-62375,62378-62379,62381 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 43577 diff changeset	576 argument should be the name of a package, in standard module format
beb7abcd6c01 Merged revisions 62350-62355,62358-62359,62364-62365,62370,62372-62375,62378-62379,62381 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 43577 diff changeset	577 (foo.bar). The resource argument should be in the form of a relative
beb7abcd6c01 Merged revisions 62350-62355,62358-62359,62364-62365,62370,62372-62375,62378-62379,62381 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 43577 diff changeset	578 filename, using '/' as the path separator. The parent directory name '..'
beb7abcd6c01 Merged revisions 62350-62355,62358-62359,62364-62365,62370,62372-62375,62378-62379,62381 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 43577 diff changeset	579 is not allowed, and nor is a rooted name (starting with a '/').
beb7abcd6c01 Merged revisions 62350-62355,62358-62359,62364-62365,62370,62372-62375,62378-62379,62381 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 43577 diff changeset	580
beb7abcd6c01 Merged revisions 62350-62355,62358-62359,62364-62365,62370,62372-62375,62378-62379,62381 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 43577 diff changeset	581 The function returns a binary string, which is the contents of the
beb7abcd6c01 Merged revisions 62350-62355,62358-62359,62364-62365,62370,62372-62375,62378-62379,62381 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 43577 diff changeset	582 specified resource.
beb7abcd6c01 Merged revisions 62350-62355,62358-62359,62364-62365,62370,62372-62375,62378-62379,62381 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 43577 diff changeset	583
beb7abcd6c01 Merged revisions 62350-62355,62358-62359,62364-62365,62370,62372-62375,62378-62379,62381 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 43577 diff changeset	584 For packages located in the filesystem, which have already been imported,
beb7abcd6c01 Merged revisions 62350-62355,62358-62359,62364-62365,62370,62372-62375,62378-62379,62381 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 43577 diff changeset	585 this is the rough equivalent of
beb7abcd6c01 Merged revisions 62350-62355,62358-62359,62364-62365,62370,62372-62375,62378-62379,62381 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 43577 diff changeset	586
beb7abcd6c01 Merged revisions 62350-62355,62358-62359,62364-62365,62370,62372-62375,62378-62379,62381 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 43577 diff changeset	587 d = os.path.dirname(sys.modules[package].__file__)
beb7abcd6c01 Merged revisions 62350-62355,62358-62359,62364-62365,62370,62372-62375,62378-62379,62381 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 43577 diff changeset	588 data = open(os.path.join(d, resource), 'rb').read()
beb7abcd6c01 Merged revisions 62350-62355,62358-62359,62364-62365,62370,62372-62375,62378-62379,62381 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 43577 diff changeset	589
beb7abcd6c01 Merged revisions 62350-62355,62358-62359,62364-62365,62370,62372-62375,62378-62379,62381 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 43577 diff changeset	590 If the package cannot be located or loaded, or it uses a PEP 302 loader
beb7abcd6c01 Merged revisions 62350-62355,62358-62359,62364-62365,62370,62372-62375,62378-62379,62381 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 43577 diff changeset	591 which does not support get_data(), then None is returned.
beb7abcd6c01 Merged revisions 62350-62355,62358-62359,62364-62365,62370,62372-62375,62378-62379,62381 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 43577 diff changeset	592 """
beb7abcd6c01 Merged revisions 62350-62355,62358-62359,62364-62365,62370,62372-62375,62378-62379,62381 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 43577 diff changeset	593
beb7abcd6c01 Merged revisions 62350-62355,62358-62359,62364-62365,62370,62372-62375,62378-62379,62381 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 43577 diff changeset	594 loader = get_loader(package)
beb7abcd6c01 Merged revisions 62350-62355,62358-62359,62364-62365,62370,62372-62375,62378-62379,62381 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 43577 diff changeset	595 if loader is None or not hasattr(loader, 'get_data'):
beb7abcd6c01 Merged revisions 62350-62355,62358-62359,62364-62365,62370,62372-62375,62378-62379,62381 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 43577 diff changeset	596 return None
beb7abcd6c01 Merged revisions 62350-62355,62358-62359,62364-62365,62370,62372-62375,62378-62379,62381 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 43577 diff changeset	597 mod = sys.modules.get(package) or loader.load_module(package)
beb7abcd6c01 Merged revisions 62350-62355,62358-62359,62364-62365,62370,62372-62375,62378-62379,62381 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 43577 diff changeset	598 if mod is None or not hasattr(mod, '__file__'):
beb7abcd6c01 Merged revisions 62350-62355,62358-62359,62364-62365,62370,62372-62375,62378-62379,62381 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 43577 diff changeset	599 return None
beb7abcd6c01 Merged revisions 62350-62355,62358-62359,62364-62365,62370,62372-62375,62378-62379,62381 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 43577 diff changeset	600
beb7abcd6c01 Merged revisions 62350-62355,62358-62359,62364-62365,62370,62372-62375,62378-62379,62381 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 43577 diff changeset	601 # Modify the resource name to be compatible with the loader.get_data
beb7abcd6c01 Merged revisions 62350-62355,62358-62359,62364-62365,62370,62372-62375,62378-62379,62381 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 43577 diff changeset	602 # signature - an os.path format "filename" starting with the dirname of
beb7abcd6c01 Merged revisions 62350-62355,62358-62359,62364-62365,62370,62372-62375,62378-62379,62381 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 43577 diff changeset	603 # the package's __file__
beb7abcd6c01 Merged revisions 62350-62355,62358-62359,62364-62365,62370,62372-62375,62378-62379,62381 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 43577 diff changeset	604 parts = resource.split('/')
beb7abcd6c01 Merged revisions 62350-62355,62358-62359,62364-62365,62370,62372-62375,62378-62379,62381 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 43577 diff changeset	605 parts.insert(0, os.path.dirname(mod.__file__))
beb7abcd6c01 Merged revisions 62350-62355,62358-62359,62364-62365,62370,62372-62375,62378-62379,62381 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 43577 diff changeset	606 resource_name = os.path.join(*parts)
beb7abcd6c01 Merged revisions 62350-62355,62358-62359,62364-62365,62370,62372-62375,62378-62379,62381 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 43577 diff changeset	607 return loader.get_data(resource_name)

Mercurial > cpython-lru_cache-weakref

annotate Lib/pkgutil.py @ 86127:a3d09c19bb93