Mercurial > cpython-lru_cache-weakref

annotate Lib/test/test_importlib/source/test_source_encoding.py @ 87690:66c1c9f32567 default tip

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Add use_weakref option to functools.lru_cache The use_weakref option makes lru_cache create weak references to the function arguments when creating the cache key, which prevents them from staying in the memory forever even after all other references to them are gone. This is a proof of concept for http://bugs.python.org/issue19859
author Radomir Dopieralski <devel@sheep.art.pl>
date Tue, 03 Dec 2013 16:02:48 +0100
parents b0f570aef6fd
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
87009
b0f570aef6fd Issue #16803: test.test_importlib.source now tests frozen and source code
Brett Cannon <brett@python.org>
parents: 85731
diff changeset
1 from .. import util
51234
4b3246c83e3f Split out support code that is specific to source tests out of
Brett Cannon <bcannon@gmail.com>
parents: 50845
diff changeset
2 from . import util as source_util
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
3
87009
b0f570aef6fd Issue #16803: test.test_importlib.source now tests frozen and source code
Brett Cannon <brett@python.org>
parents: 85731
diff changeset
4 machinery = util.import_importlib('importlib.machinery')
b0f570aef6fd Issue #16803: test.test_importlib.source now tests frozen and source code
Brett Cannon <brett@python.org>
parents: 85731
diff changeset
5
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
6 import codecs
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
7 import re
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
8 import sys
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
9 # Because sys.path gets essentially blanked, need to have unicodedata already
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
10 # imported for the parser to use.
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
11 import unicodedata
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
12 import unittest
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
13
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
14
85731
2dfe8262093c Issue #18873: The tokenize module, IDLE, 2to3, and the findnocoding.py script
Serhiy Storchaka <storchaka@gmail.com>
parents: 78201
diff changeset
15 CODING_RE = re.compile(r'^[ \t\f]*#.*coding[:=][ \t]*([-\w.]+)', re.ASCII)
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
16
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
17
87009
b0f570aef6fd Issue #16803: test.test_importlib.source now tests frozen and source code
Brett Cannon <brett@python.org>
parents: 85731
diff changeset
18 class EncodingTest:
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
19
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
20 """PEP 3120 makes UTF-8 the default encoding for source code
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
21 [default encoding].
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
22
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
23 PEP 263 specifies how that can change on a per-file basis. Either the first
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
24 or second line can contain the encoding line [encoding first line]
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
25 encoding second line]. If the file has the BOM marker it is considered UTF-8
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
26 implicitly [BOM]. If any encoding is specified it must be UTF-8, else it is
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
27 an error [BOM and utf-8][BOM conflict].
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
28
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
29 """
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
30
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
31 variable = '\u00fc'
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
32 character = '\u00c9'
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
33 source_line = "{0} = '{1}'\n".format(variable, character)
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
34 module_name = '_temp'
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
35
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
36 def run_test(self, source):
51234
4b3246c83e3f Split out support code that is specific to source tests out of
Brett Cannon <bcannon@gmail.com>
parents: 50845
diff changeset
37 with source_util.create_modules(self.module_name) as mapping:
60648
7b69e630d237 PEP 3147
Barry Warsaw <barry@python.org>
parents: 55737
diff changeset
38 with open(mapping[self.module_name], 'wb') as file:
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
39 file.write(source)
87009
b0f570aef6fd Issue #16803: test.test_importlib.source now tests frozen and source code
Brett Cannon <brett@python.org>
parents: 85731
diff changeset
40 loader = self.machinery.SourceFileLoader(self.module_name,
76303
2dd046be2c88 Issue #2377: Make importlib the implementation of __import__().
Brett Cannon <brett@python.org>
parents: 62756
diff changeset
41 mapping[self.module_name])
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
42 return loader.load_module(self.module_name)
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
43
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
44 def create_source(self, encoding):
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
45 encoding_line = "# coding={0}".format(encoding)
85731
2dfe8262093c Issue #18873: The tokenize module, IDLE, 2to3, and the findnocoding.py script
Serhiy Storchaka <storchaka@gmail.com>
parents: 78201
diff changeset
46 assert CODING_RE.match(encoding_line)
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
47 source_lines = [encoding_line.encode('utf-8')]
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
48 source_lines.append(self.source_line.encode(encoding))
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
49 return b'\n'.join(source_lines)
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
50
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
51 def test_non_obvious_encoding(self):
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
52 # Make sure that an encoding that has never been a standard one for
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
53 # Python works.
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
54 encoding_line = "# coding=koi8-r"
85731
2dfe8262093c Issue #18873: The tokenize module, IDLE, 2to3, and the findnocoding.py script
Serhiy Storchaka <storchaka@gmail.com>
parents: 78201
diff changeset
55 assert CODING_RE.match(encoding_line)
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
56 source = "{0}\na=42\n".format(encoding_line).encode("koi8-r")
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
57 self.run_test(source)
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
58
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
59 # [default encoding]
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
60 def test_default_encoding(self):
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
61 self.run_test(self.source_line.encode('utf-8'))
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
62
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
63 # [encoding first line]
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
64 def test_encoding_on_first_line(self):
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
65 encoding = 'Latin-1'
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
66 source = self.create_source(encoding)
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
67 self.run_test(source)
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
68
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
69 # [encoding second line]
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
70 def test_encoding_on_second_line(self):
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
71 source = b"#/usr/bin/python\n" + self.create_source('Latin-1')
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
72 self.run_test(source)
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
73
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
74 # [BOM]
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
75 def test_bom(self):
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
76 self.run_test(codecs.BOM_UTF8 + self.source_line.encode('utf-8'))
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
77
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
78 # [BOM and utf-8]
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
79 def test_bom_and_utf_8(self):
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
80 source = codecs.BOM_UTF8 + self.create_source('utf-8')
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
81 self.run_test(source)
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
82
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
83 # [BOM conflict]
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
84 def test_bom_conflict(self):
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
85 source = codecs.BOM_UTF8 + self.create_source('latin-1')
55737
b65ebed06eb0 Move over to using assertRaises as a context manager for importlib tests.
Brett Cannon <bcannon@gmail.com>
parents: 52142
diff changeset
86 with self.assertRaises(SyntaxError):
b65ebed06eb0 Move over to using assertRaises as a context manager for importlib tests.
Brett Cannon <bcannon@gmail.com>
parents: 52142
diff changeset
87 self.run_test(source)
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
88
87009
b0f570aef6fd Issue #16803: test.test_importlib.source now tests frozen and source code
Brett Cannon <brett@python.org>
parents: 85731
diff changeset
89 Frozen_EncodingTest, Source_EncodingTest = util.test_both(EncodingTest, machinery=machinery)
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
90
87009
b0f570aef6fd Issue #16803: test.test_importlib.source now tests frozen and source code
Brett Cannon <brett@python.org>
parents: 85731
diff changeset
91
b0f570aef6fd Issue #16803: test.test_importlib.source now tests frozen and source code
Brett Cannon <brett@python.org>
parents: 85731
diff changeset
92 class LineEndingTest:
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
93
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
94 r"""Source written with the three types of line endings (\n, \r\n, \r)
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
95 need to be readable [cr][crlf][lf]."""
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
96
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
97 def run_test(self, line_ending):
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
98 module_name = '_temp'
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
99 source_lines = [b"a = 42", b"b = -13", b'']
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
100 source = line_ending.join(source_lines)
51234
4b3246c83e3f Split out support code that is specific to source tests out of
Brett Cannon <bcannon@gmail.com>
parents: 50845
diff changeset
101 with source_util.create_modules(module_name) as mapping:
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
102 with open(mapping[module_name], 'wb') as file:
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
103 file.write(source)
87009
b0f570aef6fd Issue #16803: test.test_importlib.source now tests frozen and source code
Brett Cannon <brett@python.org>
parents: 85731
diff changeset
104 loader = self.machinery.SourceFileLoader(module_name,
62756
34d827249ce0 Make importlib.abc.SourceLoader the primary mechanism for importlib.
Brett Cannon <bcannon@gmail.com>
parents: 60648
diff changeset
105 mapping[module_name])
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
106 return loader.load_module(module_name)
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
107
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
108 # [cr]
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
109 def test_cr(self):
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
110 self.run_test(b'\r')
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
111
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
112 # [crlf]
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
113 def test_crlf(self):
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
114 self.run_test(b'\r\n')
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
115
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
116 # [lf]
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
117 def test_lf(self):
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
118 self.run_test(b'\n')
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
119
87009
b0f570aef6fd Issue #16803: test.test_importlib.source now tests frozen and source code
Brett Cannon <brett@python.org>
parents: 85731
diff changeset
120 Frozen_LineEndings, Source_LineEndings = util.test_both(LineEndingTest, machinery=machinery)
50845
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
121
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
122
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
123
aac51b70c74c Add initial implementation of importlib. See the NOTES files for what is
Brett Cannon <bcannon@gmail.com>
parents:
diff changeset
124 if __name__ == '__main__':
87009
b0f570aef6fd Issue #16803: test.test_importlib.source now tests frozen and source code
Brett Cannon <brett@python.org>
parents: 85731
diff changeset
125 unittest.main()