annotate Lib/packaging/command/upload_docs.py @ 73323:dad02a080bbc

Improve byte-compilation in packaging to be independent of -O or -B. The code I fixed to comply with PEP 3147 still had one bug: When run under python -O, some paths for pyc files would be pyo, because I called imp.cache_from_source without explicit debug_override argument in some places, and under -O that would return .pyo (this is well explained in the imp docs). Now all code (util.byte_compile, build_py, install_lib) can create .pyo files according to options given by users, without interference from the calling Python’s own optimize mode. On a related topic, I also removed the code that prevented byte compilation under python -B. The rationale is that packaging gives control over the creation of pyc files to the user with its own explicit option, and the behavior should not be changed if the calling Python happens to run with -B for whatever reason. I will argue that this is a bug fix and ask to be allowed to backport this change to distutils. Finally, I moved one nugget of information about the --compile and --optimize options from the source into the doc. It clears up a misunderstanding that I (and maybe other people) had.
author Éric Araujo <merwok@netwok.org>
date Thu, 03 Nov 2011 05:08:28 +0100
parents a400c1c087dd
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
70194
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
1 """Upload HTML documentation to a project index."""
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
2
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
3 import os
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
4 import base64
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
5 import socket
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
6 import zipfile
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
7 import logging
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
8 import http.client
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
9 import urllib.parse
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
10 from io import BytesIO
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
11
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
12 from packaging import logger
71253
c2785ed52ed4 Factor out code used by packaging commands for HTTP requests (#12169).
Éric Araujo <merwok@netwok.org>
parents: 70202
diff changeset
13 from packaging.util import (read_pypirc, DEFAULT_REPOSITORY, DEFAULT_REALM,
c2785ed52ed4 Factor out code used by packaging commands for HTTP requests (#12169).
Éric Araujo <merwok@netwok.org>
parents: 70202
diff changeset
14 encode_multipart)
70194
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
15 from packaging.errors import PackagingFileError
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
16 from packaging.command.cmd import Command
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
17
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
18
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
19 def zip_dir(directory):
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
20 """Compresses recursively contents of directory into a BytesIO object"""
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
21 destination = BytesIO()
70202
8dc80509f181 packaging: use with open() instead of try/finally: close
Victor Stinner <victor.stinner@haypocalc.com>
parents: 70194
diff changeset
22 with zipfile.ZipFile(destination, "w") as zip_file:
8dc80509f181 packaging: use with open() instead of try/finally: close
Victor Stinner <victor.stinner@haypocalc.com>
parents: 70194
diff changeset
23 for root, dirs, files in os.walk(directory):
8dc80509f181 packaging: use with open() instead of try/finally: close
Victor Stinner <victor.stinner@haypocalc.com>
parents: 70194
diff changeset
24 for name in files:
8dc80509f181 packaging: use with open() instead of try/finally: close
Victor Stinner <victor.stinner@haypocalc.com>
parents: 70194
diff changeset
25 full = os.path.join(root, name)
8dc80509f181 packaging: use with open() instead of try/finally: close
Victor Stinner <victor.stinner@haypocalc.com>
parents: 70194
diff changeset
26 relative = root[len(directory):].lstrip(os.path.sep)
8dc80509f181 packaging: use with open() instead of try/finally: close
Victor Stinner <victor.stinner@haypocalc.com>
parents: 70194
diff changeset
27 dest = os.path.join(relative, name)
8dc80509f181 packaging: use with open() instead of try/finally: close
Victor Stinner <victor.stinner@haypocalc.com>
parents: 70194
diff changeset
28 zip_file.write(full, dest)
70194
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
29 return destination
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
30
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
31
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
32 class upload_docs(Command):
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
33
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
34 description = "upload HTML documentation to PyPI"
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
35
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
36 user_options = [
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
37 ('repository=', 'r',
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
38 "repository URL [default: %s]" % DEFAULT_REPOSITORY),
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
39 ('show-response', None,
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
40 "display full response text from server"),
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
41 ('upload-dir=', None,
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
42 "directory to upload"),
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
43 ]
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
44
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
45 def initialize_options(self):
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
46 self.repository = None
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
47 self.realm = None
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
48 self.show_response = False
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
49 self.upload_dir = None
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
50 self.username = ''
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
51 self.password = ''
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
52
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
53 def finalize_options(self):
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
54 if self.repository is None:
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
55 self.repository = DEFAULT_REPOSITORY
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
56 if self.realm is None:
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
57 self.realm = DEFAULT_REALM
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
58 if self.upload_dir is None:
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
59 build = self.get_finalized_command('build')
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
60 self.upload_dir = os.path.join(build.build_base, "docs")
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
61 if not os.path.isdir(self.upload_dir):
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
62 self.upload_dir = os.path.join(build.build_base, "doc")
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
63 logger.info('Using upload directory %s', self.upload_dir)
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
64 self.verify_upload_dir(self.upload_dir)
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
65 config = read_pypirc(self.repository, self.realm)
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
66 if config != {}:
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
67 self.username = config['username']
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
68 self.password = config['password']
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
69 self.repository = config['repository']
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
70 self.realm = config['realm']
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
71
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
72 def verify_upload_dir(self, upload_dir):
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
73 self.ensure_dirname('upload_dir')
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
74 index_location = os.path.join(upload_dir, "index.html")
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
75 if not os.path.exists(index_location):
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
76 mesg = "No 'index.html found in docs directory (%s)"
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
77 raise PackagingFileError(mesg % upload_dir)
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
78
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
79 def run(self):
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
80 name = self.distribution.metadata['Name']
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
81 version = self.distribution.metadata['Version']
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
82 zip_file = zip_dir(self.upload_dir)
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
83
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
84 fields = [(':action', 'doc_upload'),
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
85 ('name', name), ('version', version)]
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
86 files = [('content', name, zip_file.getvalue())]
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
87 content_type, body = encode_multipart(fields, files)
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
88
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
89 credentials = self.username + ':' + self.password
72404
a400c1c087dd Make a number of small changes to ease the backport to distutils2
Éric Araujo <merwok@netwok.org>
parents: 71253
diff changeset
90 # FIXME should use explicit encoding
70194
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
91 auth = b"Basic " + base64.encodebytes(credentials.encode()).strip()
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
92
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
93 logger.info("Submitting documentation to %s", self.repository)
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
94
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
95 scheme, netloc, url, params, query, fragments = urllib.parse.urlparse(
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
96 self.repository)
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
97 if scheme == "http":
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
98 conn = http.client.HTTPConnection(netloc)
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
99 elif scheme == "https":
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
100 conn = http.client.HTTPSConnection(netloc)
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
101 else:
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
102 raise AssertionError("unsupported scheme %r" % scheme)
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
103
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
104 try:
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
105 conn.connect()
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
106 conn.putrequest("POST", url)
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
107 conn.putheader('Content-type', content_type)
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
108 conn.putheader('Content-length', str(len(body)))
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
109 conn.putheader('Authorization', auth)
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
110 conn.endheaders()
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
111 conn.send(body)
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
112
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
113 except socket.error as e:
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
114 logger.error(e)
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
115 return
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
116
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
117 r = conn.getresponse()
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
118
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
119 if r.status == 200:
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
120 logger.info('Server response (%s): %s', r.status, r.reason)
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
121 elif r.status == 301:
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
122 location = r.getheader('Location')
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
123 if location is None:
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
124 location = 'http://packages.python.org/%s/' % name
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
125 logger.info('Upload successful. Visit %s', location)
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
126 else:
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
127 logger.error('Upload failed (%s): %s', r.status, r.reason)
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
128
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
129 if self.show_response and logger.isEnabledFor(logging.INFO):
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
130 sep = '-' * 75
1a31c3be88b1 initial import of the packaging package in the standard library
Tarek Ziade <tarek@ziade.org>
parents:
diff changeset
131 logger.info('%s\n%s\n%s', sep, r.read().decode('utf-8'), sep)