cpython-lru_cache-weakref: Tools/scripts/highlight.py annotate

annotate Tools/scripts/highlight.py @ 84007:dbfbdf2b5c19 3.3

Fix #17967: For ftp urls CWD to target instead of hopping to each directory towards target. This fixes a bug where target is accessible, but parent directories are restricted.

author	Senthil Kumaran <senthil@uthcode.com>
date	Sat, 01 Jun 2013 08:27:06 -0700
parents	b127046831e2
children

rev	line source
77899 da4dd603030b Add syntax highlighter tool Raymond Hettinger <python@rcn.com> parents: diff changeset	1 #!/usr/bin/env python3
78014 77ca62751877 Minor algorithmic tweaks, better factoring, improved docstrings and variable names Raymond Hettinger <python@rcn.com> parents: 77944 diff changeset	2 '''Add syntax highlighting to Python source code'''
77942 1401ff261ced Add examples to the module docstring Raymond Hettinger <python@rcn.com> parents: 77941 diff changeset	3
77903 21ca7f56e12c Set title to the source filename Raymond Hettinger <python@rcn.com> parents: 77902 diff changeset	4 __author__ = 'Raymond Hettinger'
77899 da4dd603030b Add syntax highlighter tool Raymond Hettinger <python@rcn.com> parents: diff changeset	5
78049 08bfd88027bb Add simple LaTeX markup Raymond Hettinger <python@rcn.com> parents: 78021 diff changeset	6 import keyword, tokenize, cgi, re, functools
78079 425d5e89fa25 Fix builtin test and simplify the classified text tuple. Raymond Hettinger <python@rcn.com> parents: 78049 diff changeset	7 try:
425d5e89fa25 Fix builtin test and simplify the classified text tuple. Raymond Hettinger <python@rcn.com> parents: 78049 diff changeset	8 import builtins
425d5e89fa25 Fix builtin test and simplify the classified text tuple. Raymond Hettinger <python@rcn.com> parents: 78049 diff changeset	9 except ImportError:
425d5e89fa25 Fix builtin test and simplify the classified text tuple. Raymond Hettinger <python@rcn.com> parents: 78049 diff changeset	10 import __builtin__ as builtins
78049 08bfd88027bb Add simple LaTeX markup Raymond Hettinger <python@rcn.com> parents: 78021 diff changeset	11
08bfd88027bb Add simple LaTeX markup Raymond Hettinger <python@rcn.com> parents: 78021 diff changeset	12 #### Analyze Python Source #################################
77899 da4dd603030b Add syntax highlighter tool Raymond Hettinger <python@rcn.com> parents: diff changeset	13
da4dd603030b Add syntax highlighter tool Raymond Hettinger <python@rcn.com> parents: diff changeset	14 def is_builtin(s):
da4dd603030b Add syntax highlighter tool Raymond Hettinger <python@rcn.com> parents: diff changeset	15 'Return True if s is the name of a builtin'
78079 425d5e89fa25 Fix builtin test and simplify the classified text tuple. Raymond Hettinger <python@rcn.com> parents: 78049 diff changeset	16 return hasattr(builtins, s)
77899 da4dd603030b Add syntax highlighter tool Raymond Hettinger <python@rcn.com> parents: diff changeset	17
77938 1f06fe5cd4da Refactor to isolate HTML encoding step from the parsing step. Raymond Hettinger <python@rcn.com> parents: 77929 diff changeset	18 def combine_range(lines, start, end):
1f06fe5cd4da Refactor to isolate HTML encoding step from the parsing step. Raymond Hettinger <python@rcn.com> parents: 77929 diff changeset	19 'Join content from a range of lines between start and end'
77918 671894ae19a2 Do HTML escaping after the tokenization step. Raymond Hettinger <python@rcn.com> parents: 77904 diff changeset	20 (srow, scol), (erow, ecol) = start, end
671894ae19a2 Do HTML escaping after the tokenization step. Raymond Hettinger <python@rcn.com> parents: 77904 diff changeset	21 if srow == erow:
78014 77ca62751877 Minor algorithmic tweaks, better factoring, improved docstrings and variable names Raymond Hettinger <python@rcn.com> parents: 77944 diff changeset	22 return lines[srow-1][scol:ecol], end
77ca62751877 Minor algorithmic tweaks, better factoring, improved docstrings and variable names Raymond Hettinger <python@rcn.com> parents: 77944 diff changeset	23 rows = [lines[srow-1][scol:]] + lines[srow: erow-1] + [lines[erow-1][:ecol]]
77938 1f06fe5cd4da Refactor to isolate HTML encoding step from the parsing step. Raymond Hettinger <python@rcn.com> parents: 77929 diff changeset	24 return ''.join(rows), end
77918 671894ae19a2 Do HTML escaping after the tokenization step. Raymond Hettinger <python@rcn.com> parents: 77904 diff changeset	25
78014 77ca62751877 Minor algorithmic tweaks, better factoring, improved docstrings and variable names Raymond Hettinger <python@rcn.com> parents: 77944 diff changeset	26 def analyze_python(source):
77ca62751877 Minor algorithmic tweaks, better factoring, improved docstrings and variable names Raymond Hettinger <python@rcn.com> parents: 77944 diff changeset	27 '''Generate and classify chunks of Python for syntax highlighting.
78079 425d5e89fa25 Fix builtin test and simplify the classified text tuple. Raymond Hettinger <python@rcn.com> parents: 78049 diff changeset	28 Yields tuples in the form: (category, categorized_text).
78014 77ca62751877 Minor algorithmic tweaks, better factoring, improved docstrings and variable names Raymond Hettinger <python@rcn.com> parents: 77944 diff changeset	29 '''
77929 d7bed032520b Fix keyword argument Raymond Hettinger <python@rcn.com> parents: 77928 diff changeset	30 lines = source.splitlines(True)
77918 671894ae19a2 Do HTML escaping after the tokenization step. Raymond Hettinger <python@rcn.com> parents: 77904 diff changeset	31 lines.append('')
77899 da4dd603030b Add syntax highlighter tool Raymond Hettinger <python@rcn.com> parents: diff changeset	32 readline = functools.partial(next, iter(lines), '')
da4dd603030b Add syntax highlighter tool Raymond Hettinger <python@rcn.com> parents: diff changeset	33 kind = tok_str = ''
da4dd603030b Add syntax highlighter tool Raymond Hettinger <python@rcn.com> parents: diff changeset	34 tok_type = tokenize.COMMENT
77918 671894ae19a2 Do HTML escaping after the tokenization step. Raymond Hettinger <python@rcn.com> parents: 77904 diff changeset	35 written = (1, 0)
77899 da4dd603030b Add syntax highlighter tool Raymond Hettinger <python@rcn.com> parents: diff changeset	36 for tok in tokenize.generate_tokens(readline):
da4dd603030b Add syntax highlighter tool Raymond Hettinger <python@rcn.com> parents: diff changeset	37 prev_tok_type, prev_tok_str = tok_type, tok_str
da4dd603030b Add syntax highlighter tool Raymond Hettinger <python@rcn.com> parents: diff changeset	38 tok_type, tok_str, (srow, scol), (erow, ecol), logical_lineno = tok
77928 4d320cd4c6fe Minor cleanups Raymond Hettinger <python@rcn.com> parents: 77920 diff changeset	39 kind = ''
77899 da4dd603030b Add syntax highlighter tool Raymond Hettinger <python@rcn.com> parents: diff changeset	40 if tok_type == tokenize.COMMENT:
da4dd603030b Add syntax highlighter tool Raymond Hettinger <python@rcn.com> parents: diff changeset	41 kind = 'comment'
78079 425d5e89fa25 Fix builtin test and simplify the classified text tuple. Raymond Hettinger <python@rcn.com> parents: 78049 diff changeset	42 elif tok_type == tokenize.OP and tok_str[:1] not in '{}[](),.:;@':
77899 da4dd603030b Add syntax highlighter tool Raymond Hettinger <python@rcn.com> parents: diff changeset	43 kind = 'operator'
da4dd603030b Add syntax highlighter tool Raymond Hettinger <python@rcn.com> parents: diff changeset	44 elif tok_type == tokenize.STRING:
da4dd603030b Add syntax highlighter tool Raymond Hettinger <python@rcn.com> parents: diff changeset	45 kind = 'string'
da4dd603030b Add syntax highlighter tool Raymond Hettinger <python@rcn.com> parents: diff changeset	46 if prev_tok_type == tokenize.INDENT or scol==0:
da4dd603030b Add syntax highlighter tool Raymond Hettinger <python@rcn.com> parents: diff changeset	47 kind = 'docstring'
da4dd603030b Add syntax highlighter tool Raymond Hettinger <python@rcn.com> parents: diff changeset	48 elif tok_type == tokenize.NAME:
da4dd603030b Add syntax highlighter tool Raymond Hettinger <python@rcn.com> parents: diff changeset	49 if tok_str in ('def', 'class', 'import', 'from'):
da4dd603030b Add syntax highlighter tool Raymond Hettinger <python@rcn.com> parents: diff changeset	50 kind = 'definition'
da4dd603030b Add syntax highlighter tool Raymond Hettinger <python@rcn.com> parents: diff changeset	51 elif prev_tok_str in ('def', 'class'):
da4dd603030b Add syntax highlighter tool Raymond Hettinger <python@rcn.com> parents: diff changeset	52 kind = 'defname'
da4dd603030b Add syntax highlighter tool Raymond Hettinger <python@rcn.com> parents: diff changeset	53 elif keyword.iskeyword(tok_str):
da4dd603030b Add syntax highlighter tool Raymond Hettinger <python@rcn.com> parents: diff changeset	54 kind = 'keyword'
da4dd603030b Add syntax highlighter tool Raymond Hettinger <python@rcn.com> parents: diff changeset	55 elif is_builtin(tok_str) and prev_tok_str != '.':
da4dd603030b Add syntax highlighter tool Raymond Hettinger <python@rcn.com> parents: diff changeset	56 kind = 'builtin'
78014 77ca62751877 Minor algorithmic tweaks, better factoring, improved docstrings and variable names Raymond Hettinger <python@rcn.com> parents: 77944 diff changeset	57 if kind:
78079 425d5e89fa25 Fix builtin test and simplify the classified text tuple. Raymond Hettinger <python@rcn.com> parents: 78049 diff changeset	58 text, written = combine_range(lines, written, (srow, scol))
425d5e89fa25 Fix builtin test and simplify the classified text tuple. Raymond Hettinger <python@rcn.com> parents: 78049 diff changeset	59 yield '', text
78253 b127046831e2 Simplify the LaTeX section (only three escapes are needed for alltt) Raymond Hettinger <python@rcn.com> parents: 78103 diff changeset	60 text, written = tok_str, (erow, ecol)
78079 425d5e89fa25 Fix builtin test and simplify the classified text tuple. Raymond Hettinger <python@rcn.com> parents: 78049 diff changeset	61 yield kind, text
78014 77ca62751877 Minor algorithmic tweaks, better factoring, improved docstrings and variable names Raymond Hettinger <python@rcn.com> parents: 77944 diff changeset	62 line_upto_token, written = combine_range(lines, written, (erow, ecol))
78079 425d5e89fa25 Fix builtin test and simplify the classified text tuple. Raymond Hettinger <python@rcn.com> parents: 78049 diff changeset	63 yield '', line_upto_token
77938 1f06fe5cd4da Refactor to isolate HTML encoding step from the parsing step. Raymond Hettinger <python@rcn.com> parents: 77929 diff changeset	64
78049 08bfd88027bb Add simple LaTeX markup Raymond Hettinger <python@rcn.com> parents: 78021 diff changeset	65 #### Raw Output ###########################################
08bfd88027bb Add simple LaTeX markup Raymond Hettinger <python@rcn.com> parents: 78021 diff changeset	66
08bfd88027bb Add simple LaTeX markup Raymond Hettinger <python@rcn.com> parents: 78021 diff changeset	67 def raw_highlight(classified_text):
08bfd88027bb Add simple LaTeX markup Raymond Hettinger <python@rcn.com> parents: 78021 diff changeset	68 'Straight text display of text classifications'
08bfd88027bb Add simple LaTeX markup Raymond Hettinger <python@rcn.com> parents: 78021 diff changeset	69 result = []
78079 425d5e89fa25 Fix builtin test and simplify the classified text tuple. Raymond Hettinger <python@rcn.com> parents: 78049 diff changeset	70 for kind, text in classified_text:
425d5e89fa25 Fix builtin test and simplify the classified text tuple. Raymond Hettinger <python@rcn.com> parents: 78049 diff changeset	71 result.append('%15s: %r\n' % (kind or 'plain', text))
78049 08bfd88027bb Add simple LaTeX markup Raymond Hettinger <python@rcn.com> parents: 78021 diff changeset	72 return ''.join(result)
08bfd88027bb Add simple LaTeX markup Raymond Hettinger <python@rcn.com> parents: 78021 diff changeset	73
08bfd88027bb Add simple LaTeX markup Raymond Hettinger <python@rcn.com> parents: 78021 diff changeset	74 #### ANSI Output ###########################################
08bfd88027bb Add simple LaTeX markup Raymond Hettinger <python@rcn.com> parents: 78021 diff changeset	75
77940 980de5b47d0d Add support for ANSI escape sequences Raymond Hettinger <python@rcn.com> parents: 77938 diff changeset	76 default_ansi = {
78014 77ca62751877 Minor algorithmic tweaks, better factoring, improved docstrings and variable names Raymond Hettinger <python@rcn.com> parents: 77944 diff changeset	77 'comment': ('\033[0;31m', '\033[0m'),
77ca62751877 Minor algorithmic tweaks, better factoring, improved docstrings and variable names Raymond Hettinger <python@rcn.com> parents: 77944 diff changeset	78 'string': ('\033[0;32m', '\033[0m'),
77ca62751877 Minor algorithmic tweaks, better factoring, improved docstrings and variable names Raymond Hettinger <python@rcn.com> parents: 77944 diff changeset	79 'docstring': ('\033[0;32m', '\033[0m'),
77ca62751877 Minor algorithmic tweaks, better factoring, improved docstrings and variable names Raymond Hettinger <python@rcn.com> parents: 77944 diff changeset	80 'keyword': ('\033[0;33m', '\033[0m'),
77ca62751877 Minor algorithmic tweaks, better factoring, improved docstrings and variable names Raymond Hettinger <python@rcn.com> parents: 77944 diff changeset	81 'builtin': ('\033[0;35m', '\033[0m'),
77ca62751877 Minor algorithmic tweaks, better factoring, improved docstrings and variable names Raymond Hettinger <python@rcn.com> parents: 77944 diff changeset	82 'definition': ('\033[0;33m', '\033[0m'),
77ca62751877 Minor algorithmic tweaks, better factoring, improved docstrings and variable names Raymond Hettinger <python@rcn.com> parents: 77944 diff changeset	83 'defname': ('\033[0;34m', '\033[0m'),
77ca62751877 Minor algorithmic tweaks, better factoring, improved docstrings and variable names Raymond Hettinger <python@rcn.com> parents: 77944 diff changeset	84 'operator': ('\033[0;33m', '\033[0m'),
77940 980de5b47d0d Add support for ANSI escape sequences Raymond Hettinger <python@rcn.com> parents: 77938 diff changeset	85 }
980de5b47d0d Add support for ANSI escape sequences Raymond Hettinger <python@rcn.com> parents: 77938 diff changeset	86
78014 77ca62751877 Minor algorithmic tweaks, better factoring, improved docstrings and variable names Raymond Hettinger <python@rcn.com> parents: 77944 diff changeset	87 def ansi_highlight(classified_text, colors=default_ansi):
77ca62751877 Minor algorithmic tweaks, better factoring, improved docstrings and variable names Raymond Hettinger <python@rcn.com> parents: 77944 diff changeset	88 'Add syntax highlighting to source code using ANSI escape sequences'
77940 980de5b47d0d Add support for ANSI escape sequences Raymond Hettinger <python@rcn.com> parents: 77938 diff changeset	89 # http://en.wikipedia.org/wiki/ANSI_escape_code
980de5b47d0d Add support for ANSI escape sequences Raymond Hettinger <python@rcn.com> parents: 77938 diff changeset	90 result = []
78079 425d5e89fa25 Fix builtin test and simplify the classified text tuple. Raymond Hettinger <python@rcn.com> parents: 78049 diff changeset	91 for kind, text in classified_text:
78014 77ca62751877 Minor algorithmic tweaks, better factoring, improved docstrings and variable names Raymond Hettinger <python@rcn.com> parents: 77944 diff changeset	92 opener, closer = colors.get(kind, ('', ''))
78079 425d5e89fa25 Fix builtin test and simplify the classified text tuple. Raymond Hettinger <python@rcn.com> parents: 78049 diff changeset	93 result += [opener, text, closer]
77940 980de5b47d0d Add support for ANSI escape sequences Raymond Hettinger <python@rcn.com> parents: 77938 diff changeset	94 return ''.join(result)
980de5b47d0d Add support for ANSI escape sequences Raymond Hettinger <python@rcn.com> parents: 77938 diff changeset	95
78049 08bfd88027bb Add simple LaTeX markup Raymond Hettinger <python@rcn.com> parents: 78021 diff changeset	96 #### HTML Output ###########################################
08bfd88027bb Add simple LaTeX markup Raymond Hettinger <python@rcn.com> parents: 78021 diff changeset	97
78014 77ca62751877 Minor algorithmic tweaks, better factoring, improved docstrings and variable names Raymond Hettinger <python@rcn.com> parents: 77944 diff changeset	98 def html_highlight(classified_text,opener='<pre class="python">\n', closer='</pre>\n'):
77ca62751877 Minor algorithmic tweaks, better factoring, improved docstrings and variable names Raymond Hettinger <python@rcn.com> parents: 77944 diff changeset	99 'Convert classified text to an HTML fragment'
77ca62751877 Minor algorithmic tweaks, better factoring, improved docstrings and variable names Raymond Hettinger <python@rcn.com> parents: 77944 diff changeset	100 result = [opener]
78079 425d5e89fa25 Fix builtin test and simplify the classified text tuple. Raymond Hettinger <python@rcn.com> parents: 78049 diff changeset	101 for kind, text in classified_text:
77899 da4dd603030b Add syntax highlighter tool Raymond Hettinger <python@rcn.com> parents: diff changeset	102 if kind:
78079 425d5e89fa25 Fix builtin test and simplify the classified text tuple. Raymond Hettinger <python@rcn.com> parents: 78049 diff changeset	103 result.append('<span class="%s">' % kind)
425d5e89fa25 Fix builtin test and simplify the classified text tuple. Raymond Hettinger <python@rcn.com> parents: 78049 diff changeset	104 result.append(cgi.escape(text))
425d5e89fa25 Fix builtin test and simplify the classified text tuple. Raymond Hettinger <python@rcn.com> parents: 78049 diff changeset	105 if kind:
425d5e89fa25 Fix builtin test and simplify the classified text tuple. Raymond Hettinger <python@rcn.com> parents: 78049 diff changeset	106 result.append('</span>')
425d5e89fa25 Fix builtin test and simplify the classified text tuple. Raymond Hettinger <python@rcn.com> parents: 78049 diff changeset	107 result.append(closer)
77918 671894ae19a2 Do HTML escaping after the tokenization step. Raymond Hettinger <python@rcn.com> parents: 77904 diff changeset	108 return ''.join(result)
77899 da4dd603030b Add syntax highlighter tool Raymond Hettinger <python@rcn.com> parents: diff changeset	109
da4dd603030b Add syntax highlighter tool Raymond Hettinger <python@rcn.com> parents: diff changeset	110 default_css = {
da4dd603030b Add syntax highlighter tool Raymond Hettinger <python@rcn.com> parents: diff changeset	111 '.comment': '{color: crimson;}',
da4dd603030b Add syntax highlighter tool Raymond Hettinger <python@rcn.com> parents: diff changeset	112 '.string': '{color: forestgreen;}',
77938 1f06fe5cd4da Refactor to isolate HTML encoding step from the parsing step. Raymond Hettinger <python@rcn.com> parents: 77929 diff changeset	113 '.docstring': '{color: forestgreen; font-style:italic;}',
77899 da4dd603030b Add syntax highlighter tool Raymond Hettinger <python@rcn.com> parents: diff changeset	114 '.keyword': '{color: darkorange;}',
da4dd603030b Add syntax highlighter tool Raymond Hettinger <python@rcn.com> parents: diff changeset	115 '.builtin': '{color: purple;}',
da4dd603030b Add syntax highlighter tool Raymond Hettinger <python@rcn.com> parents: diff changeset	116 '.definition': '{color: darkorange; font-weight:bold;}',
da4dd603030b Add syntax highlighter tool Raymond Hettinger <python@rcn.com> parents: diff changeset	117 '.defname': '{color: blue;}',
da4dd603030b Add syntax highlighter tool Raymond Hettinger <python@rcn.com> parents: diff changeset	118 '.operator': '{color: brown;}',
da4dd603030b Add syntax highlighter tool Raymond Hettinger <python@rcn.com> parents: diff changeset	119 }
da4dd603030b Add syntax highlighter tool Raymond Hettinger <python@rcn.com> parents: diff changeset	120
da4dd603030b Add syntax highlighter tool Raymond Hettinger <python@rcn.com> parents: diff changeset	121 default_html = '''\
77902 d02588a26057 Small cleanups Raymond Hettinger <python@rcn.com> parents: 77901 diff changeset	122 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN"
d02588a26057 Small cleanups Raymond Hettinger <python@rcn.com> parents: 77901 diff changeset	123 "http://www.w3.org/TR/html4/strict.dtd">
d02588a26057 Small cleanups Raymond Hettinger <python@rcn.com> parents: 77901 diff changeset	124 <html>
d02588a26057 Small cleanups Raymond Hettinger <python@rcn.com> parents: 77901 diff changeset	125 <head>
d02588a26057 Small cleanups Raymond Hettinger <python@rcn.com> parents: 77901 diff changeset	126 <meta http-equiv="Content-type" content="text/html;charset=UTF-8">
77920 2367ec12da2d Use new-style string formatting for the HTML template Raymond Hettinger <python@rcn.com> parents: 77919 diff changeset	127 <title> {title} </title>
77902 d02588a26057 Small cleanups Raymond Hettinger <python@rcn.com> parents: 77901 diff changeset	128 <style type="text/css">
77920 2367ec12da2d Use new-style string formatting for the HTML template Raymond Hettinger <python@rcn.com> parents: 77919 diff changeset	129 {css}
77902 d02588a26057 Small cleanups Raymond Hettinger <python@rcn.com> parents: 77901 diff changeset	130 </style>
d02588a26057 Small cleanups Raymond Hettinger <python@rcn.com> parents: 77901 diff changeset	131 </head>
77899 da4dd603030b Add syntax highlighter tool Raymond Hettinger <python@rcn.com> parents: diff changeset	132 <body>
77920 2367ec12da2d Use new-style string formatting for the HTML template Raymond Hettinger <python@rcn.com> parents: 77919 diff changeset	133 {body}
77902 d02588a26057 Small cleanups Raymond Hettinger <python@rcn.com> parents: 77901 diff changeset	134 </body>
d02588a26057 Small cleanups Raymond Hettinger <python@rcn.com> parents: 77901 diff changeset	135 </html>
77899 da4dd603030b Add syntax highlighter tool Raymond Hettinger <python@rcn.com> parents: diff changeset	136 '''
da4dd603030b Add syntax highlighter tool Raymond Hettinger <python@rcn.com> parents: diff changeset	137
78014 77ca62751877 Minor algorithmic tweaks, better factoring, improved docstrings and variable names Raymond Hettinger <python@rcn.com> parents: 77944 diff changeset	138 def build_html_page(classified_text, title='python',
77ca62751877 Minor algorithmic tweaks, better factoring, improved docstrings and variable names Raymond Hettinger <python@rcn.com> parents: 77944 diff changeset	139 css=default_css, html=default_html):
77ca62751877 Minor algorithmic tweaks, better factoring, improved docstrings and variable names Raymond Hettinger <python@rcn.com> parents: 77944 diff changeset	140 'Create a complete HTML page with colorized source code'
77902 d02588a26057 Small cleanups Raymond Hettinger <python@rcn.com> parents: 77901 diff changeset	141 css_str = '\n'.join(['%s %s' % item for item in css.items()])
78014 77ca62751877 Minor algorithmic tweaks, better factoring, improved docstrings and variable names Raymond Hettinger <python@rcn.com> parents: 77944 diff changeset	142 result = html_highlight(classified_text)
77903 21ca7f56e12c Set title to the source filename Raymond Hettinger <python@rcn.com> parents: 77902 diff changeset	143 title = cgi.escape(title)
77920 2367ec12da2d Use new-style string formatting for the HTML template Raymond Hettinger <python@rcn.com> parents: 77919 diff changeset	144 return html.format(title=title, css=css_str, body=result)
77899 da4dd603030b Add syntax highlighter tool Raymond Hettinger <python@rcn.com> parents: diff changeset	145
78049 08bfd88027bb Add simple LaTeX markup Raymond Hettinger <python@rcn.com> parents: 78021 diff changeset	146 #### LaTeX Output ##########################################
08bfd88027bb Add simple LaTeX markup Raymond Hettinger <python@rcn.com> parents: 78021 diff changeset	147
78103 592307606f51 Fix-up latex styling Raymond Hettinger <python@rcn.com> parents: 78079 diff changeset	148 default_latex_commands = {
592307606f51 Fix-up latex styling Raymond Hettinger <python@rcn.com> parents: 78079 diff changeset	149 'comment': '{\color{red}#1}',
592307606f51 Fix-up latex styling Raymond Hettinger <python@rcn.com> parents: 78079 diff changeset	150 'string': '{\color{ForestGreen}#1}',
592307606f51 Fix-up latex styling Raymond Hettinger <python@rcn.com> parents: 78079 diff changeset	151 'docstring': '{\emph{\color{ForestGreen}#1}}',
592307606f51 Fix-up latex styling Raymond Hettinger <python@rcn.com> parents: 78079 diff changeset	152 'keyword': '{\color{orange}#1}',
592307606f51 Fix-up latex styling Raymond Hettinger <python@rcn.com> parents: 78079 diff changeset	153 'builtin': '{\color{purple}#1}',
592307606f51 Fix-up latex styling Raymond Hettinger <python@rcn.com> parents: 78079 diff changeset	154 'definition': '{\color{orange}#1}',
592307606f51 Fix-up latex styling Raymond Hettinger <python@rcn.com> parents: 78079 diff changeset	155 'defname': '{\color{blue}#1}',
592307606f51 Fix-up latex styling Raymond Hettinger <python@rcn.com> parents: 78079 diff changeset	156 'operator': '{\color{brown}#1}',
78049 08bfd88027bb Add simple LaTeX markup Raymond Hettinger <python@rcn.com> parents: 78021 diff changeset	157 }
08bfd88027bb Add simple LaTeX markup Raymond Hettinger <python@rcn.com> parents: 78021 diff changeset	158
08bfd88027bb Add simple LaTeX markup Raymond Hettinger <python@rcn.com> parents: 78021 diff changeset	159 default_latex_document = r'''
08bfd88027bb Add simple LaTeX markup Raymond Hettinger <python@rcn.com> parents: 78021 diff changeset	160 \documentclass{article}
08bfd88027bb Add simple LaTeX markup Raymond Hettinger <python@rcn.com> parents: 78021 diff changeset	161 \usepackage{alltt}
78103 592307606f51 Fix-up latex styling Raymond Hettinger <python@rcn.com> parents: 78079 diff changeset	162 \usepackage{upquote}
78049 08bfd88027bb Add simple LaTeX markup Raymond Hettinger <python@rcn.com> parents: 78021 diff changeset	163 \usepackage{color}
08bfd88027bb Add simple LaTeX markup Raymond Hettinger <python@rcn.com> parents: 78021 diff changeset	164 \usepackage[usenames,dvipsnames]{xcolor}
08bfd88027bb Add simple LaTeX markup Raymond Hettinger <python@rcn.com> parents: 78021 diff changeset	165 \usepackage[cm]{fullpage}
78103 592307606f51 Fix-up latex styling Raymond Hettinger <python@rcn.com> parents: 78079 diff changeset	166 %(macros)s
78049 08bfd88027bb Add simple LaTeX markup Raymond Hettinger <python@rcn.com> parents: 78021 diff changeset	167 \begin{document}
08bfd88027bb Add simple LaTeX markup Raymond Hettinger <python@rcn.com> parents: 78021 diff changeset	168 \center{\LARGE{%(title)s}}
08bfd88027bb Add simple LaTeX markup Raymond Hettinger <python@rcn.com> parents: 78021 diff changeset	169 \begin{alltt}
08bfd88027bb Add simple LaTeX markup Raymond Hettinger <python@rcn.com> parents: 78021 diff changeset	170 %(body)s
08bfd88027bb Add simple LaTeX markup Raymond Hettinger <python@rcn.com> parents: 78021 diff changeset	171 \end{alltt}
08bfd88027bb Add simple LaTeX markup Raymond Hettinger <python@rcn.com> parents: 78021 diff changeset	172 \end{document}
08bfd88027bb Add simple LaTeX markup Raymond Hettinger <python@rcn.com> parents: 78021 diff changeset	173 '''
08bfd88027bb Add simple LaTeX markup Raymond Hettinger <python@rcn.com> parents: 78021 diff changeset	174
78253 b127046831e2 Simplify the LaTeX section (only three escapes are needed for alltt) Raymond Hettinger <python@rcn.com> parents: 78103 diff changeset	175 def alltt_escape(s):
b127046831e2 Simplify the LaTeX section (only three escapes are needed for alltt) Raymond Hettinger <python@rcn.com> parents: 78103 diff changeset	176 'Replace backslash and braces with their escaped equivalents'
b127046831e2 Simplify the LaTeX section (only three escapes are needed for alltt) Raymond Hettinger <python@rcn.com> parents: 78103 diff changeset	177 xlat = {'{': r'\{', '}': r'\}', '\\': r'\textbackslash{}'}
b127046831e2 Simplify the LaTeX section (only three escapes are needed for alltt) Raymond Hettinger <python@rcn.com> parents: 78103 diff changeset	178 return re.sub(r'[\\{}]', lambda mo: xlat[mo.group()], s)
78049 08bfd88027bb Add simple LaTeX markup Raymond Hettinger <python@rcn.com> parents: 78021 diff changeset	179
08bfd88027bb Add simple LaTeX markup Raymond Hettinger <python@rcn.com> parents: 78021 diff changeset	180 def latex_highlight(classified_text, title = 'python',
78103 592307606f51 Fix-up latex styling Raymond Hettinger <python@rcn.com> parents: 78079 diff changeset	181 commands = default_latex_commands,
78049 08bfd88027bb Add simple LaTeX markup Raymond Hettinger <python@rcn.com> parents: 78021 diff changeset	182 document = default_latex_document):
08bfd88027bb Add simple LaTeX markup Raymond Hettinger <python@rcn.com> parents: 78021 diff changeset	183 'Create a complete LaTeX document with colorized source code'
78103 592307606f51 Fix-up latex styling Raymond Hettinger <python@rcn.com> parents: 78079 diff changeset	184 macros = '\n'.join(r'\newcommand{\py%s}[1]{%s}' % c for c in commands.items())
78049 08bfd88027bb Add simple LaTeX markup Raymond Hettinger <python@rcn.com> parents: 78021 diff changeset	185 result = []
78079 425d5e89fa25 Fix builtin test and simplify the classified text tuple. Raymond Hettinger <python@rcn.com> parents: 78049 diff changeset	186 for kind, text in classified_text:
78049 08bfd88027bb Add simple LaTeX markup Raymond Hettinger <python@rcn.com> parents: 78021 diff changeset	187 if kind:
78103 592307606f51 Fix-up latex styling Raymond Hettinger <python@rcn.com> parents: 78079 diff changeset	188 result.append(r'\py%s{' % kind)
78253 b127046831e2 Simplify the LaTeX section (only three escapes are needed for alltt) Raymond Hettinger <python@rcn.com> parents: 78103 diff changeset	189 result.append(alltt_escape(text))
78079 425d5e89fa25 Fix builtin test and simplify the classified text tuple. Raymond Hettinger <python@rcn.com> parents: 78049 diff changeset	190 if kind:
425d5e89fa25 Fix builtin test and simplify the classified text tuple. Raymond Hettinger <python@rcn.com> parents: 78049 diff changeset	191 result.append('}')
78103 592307606f51 Fix-up latex styling Raymond Hettinger <python@rcn.com> parents: 78079 diff changeset	192 return default_latex_document % dict(title=title, macros=macros, body=''.join(result))
78049 08bfd88027bb Add simple LaTeX markup Raymond Hettinger <python@rcn.com> parents: 78021 diff changeset	193
77899 da4dd603030b Add syntax highlighter tool Raymond Hettinger <python@rcn.com> parents: diff changeset	194
da4dd603030b Add syntax highlighter tool Raymond Hettinger <python@rcn.com> parents: diff changeset	195 if __name__ == '__main__':
78014 77ca62751877 Minor algorithmic tweaks, better factoring, improved docstrings and variable names Raymond Hettinger <python@rcn.com> parents: 77944 diff changeset	196 import sys, argparse, webbrowser, os, textwrap
77899 da4dd603030b Add syntax highlighter tool Raymond Hettinger <python@rcn.com> parents: diff changeset	197
da4dd603030b Add syntax highlighter tool Raymond Hettinger <python@rcn.com> parents: diff changeset	198 parser = argparse.ArgumentParser(
78014 77ca62751877 Minor algorithmic tweaks, better factoring, improved docstrings and variable names Raymond Hettinger <python@rcn.com> parents: 77944 diff changeset	199 description = 'Add syntax highlighting to Python source code',
77ca62751877 Minor algorithmic tweaks, better factoring, improved docstrings and variable names Raymond Hettinger <python@rcn.com> parents: 77944 diff changeset	200 formatter_class=argparse.RawDescriptionHelpFormatter,
77ca62751877 Minor algorithmic tweaks, better factoring, improved docstrings and variable names Raymond Hettinger <python@rcn.com> parents: 77944 diff changeset	201 epilog = textwrap.dedent('''
77ca62751877 Minor algorithmic tweaks, better factoring, improved docstrings and variable names Raymond Hettinger <python@rcn.com> parents: 77944 diff changeset	202 examples:
77ca62751877 Minor algorithmic tweaks, better factoring, improved docstrings and variable names Raymond Hettinger <python@rcn.com> parents: 77944 diff changeset	203
77ca62751877 Minor algorithmic tweaks, better factoring, improved docstrings and variable names Raymond Hettinger <python@rcn.com> parents: 77944 diff changeset	204 # Show syntax highlighted code in the terminal window
77ca62751877 Minor algorithmic tweaks, better factoring, improved docstrings and variable names Raymond Hettinger <python@rcn.com> parents: 77944 diff changeset	205 $ ./highlight.py myfile.py
77ca62751877 Minor algorithmic tweaks, better factoring, improved docstrings and variable names Raymond Hettinger <python@rcn.com> parents: 77944 diff changeset	206
77ca62751877 Minor algorithmic tweaks, better factoring, improved docstrings and variable names Raymond Hettinger <python@rcn.com> parents: 77944 diff changeset	207 # Colorize myfile.py and display in a browser
77ca62751877 Minor algorithmic tweaks, better factoring, improved docstrings and variable names Raymond Hettinger <python@rcn.com> parents: 77944 diff changeset	208 $ ./highlight.py -b myfile.py
77ca62751877 Minor algorithmic tweaks, better factoring, improved docstrings and variable names Raymond Hettinger <python@rcn.com> parents: 77944 diff changeset	209
77ca62751877 Minor algorithmic tweaks, better factoring, improved docstrings and variable names Raymond Hettinger <python@rcn.com> parents: 77944 diff changeset	210 # Create an HTML section to embed in an existing webpage
77ca62751877 Minor algorithmic tweaks, better factoring, improved docstrings and variable names Raymond Hettinger <python@rcn.com> parents: 77944 diff changeset	211 ./highlight.py -s myfile.py
77ca62751877 Minor algorithmic tweaks, better factoring, improved docstrings and variable names Raymond Hettinger <python@rcn.com> parents: 77944 diff changeset	212
77ca62751877 Minor algorithmic tweaks, better factoring, improved docstrings and variable names Raymond Hettinger <python@rcn.com> parents: 77944 diff changeset	213 # Create a complete HTML file
77ca62751877 Minor algorithmic tweaks, better factoring, improved docstrings and variable names Raymond Hettinger <python@rcn.com> parents: 77944 diff changeset	214 $ ./highlight.py -c myfile.py > myfile.html
78049 08bfd88027bb Add simple LaTeX markup Raymond Hettinger <python@rcn.com> parents: 78021 diff changeset	215
08bfd88027bb Add simple LaTeX markup Raymond Hettinger <python@rcn.com> parents: 78021 diff changeset	216 # Create a PDF using LaTeX
08bfd88027bb Add simple LaTeX markup Raymond Hettinger <python@rcn.com> parents: 78021 diff changeset	217 $ ./highlight.py -l myfile.py \| pdflatex
08bfd88027bb Add simple LaTeX markup Raymond Hettinger <python@rcn.com> parents: 78021 diff changeset	218
78014 77ca62751877 Minor algorithmic tweaks, better factoring, improved docstrings and variable names Raymond Hettinger <python@rcn.com> parents: 77944 diff changeset	219 '''))
77928 4d320cd4c6fe Minor cleanups Raymond Hettinger <python@rcn.com> parents: 77920 diff changeset	220 parser.add_argument('sourcefile', metavar = 'SOURCEFILE',
78014 77ca62751877 Minor algorithmic tweaks, better factoring, improved docstrings and variable names Raymond Hettinger <python@rcn.com> parents: 77944 diff changeset	221 help = 'file containing Python sourcecode')
77899 da4dd603030b Add syntax highlighter tool Raymond Hettinger <python@rcn.com> parents: diff changeset	222 parser.add_argument('-b', '--browser', action = 'store_true',
da4dd603030b Add syntax highlighter tool Raymond Hettinger <python@rcn.com> parents: diff changeset	223 help = 'launch a browser to show results')
77943 9fe720fe6057 Make ANSI the default output style Raymond Hettinger <python@rcn.com> parents: 77942 diff changeset	224 parser.add_argument('-c', '--complete', action = 'store_true',
9fe720fe6057 Make ANSI the default output style Raymond Hettinger <python@rcn.com> parents: 77942 diff changeset	225 help = 'build a complete html webpage')
78049 08bfd88027bb Add simple LaTeX markup Raymond Hettinger <python@rcn.com> parents: 78021 diff changeset	226 parser.add_argument('-l', '--latex', action = 'store_true',
08bfd88027bb Add simple LaTeX markup Raymond Hettinger <python@rcn.com> parents: 78021 diff changeset	227 help = 'build a LaTeX document')
08bfd88027bb Add simple LaTeX markup Raymond Hettinger <python@rcn.com> parents: 78021 diff changeset	228 parser.add_argument('-r', '--raw', action = 'store_true',
08bfd88027bb Add simple LaTeX markup Raymond Hettinger <python@rcn.com> parents: 78021 diff changeset	229 help = 'raw parse of categorized text')
77928 4d320cd4c6fe Minor cleanups Raymond Hettinger <python@rcn.com> parents: 77920 diff changeset	230 parser.add_argument('-s', '--section', action = 'store_true',
4d320cd4c6fe Minor cleanups Raymond Hettinger <python@rcn.com> parents: 77920 diff changeset	231 help = 'show an HTML section rather than a complete webpage')
77899 da4dd603030b Add syntax highlighter tool Raymond Hettinger <python@rcn.com> parents: diff changeset	232 args = parser.parse_args()
77940 980de5b47d0d Add support for ANSI escape sequences Raymond Hettinger <python@rcn.com> parents: 77938 diff changeset	233
77943 9fe720fe6057 Make ANSI the default output style Raymond Hettinger <python@rcn.com> parents: 77942 diff changeset	234 if args.section and (args.browser or args.complete):
77928 4d320cd4c6fe Minor cleanups Raymond Hettinger <python@rcn.com> parents: 77920 diff changeset	235 parser.error('The -s/--section option is incompatible with '
77943 9fe720fe6057 Make ANSI the default output style Raymond Hettinger <python@rcn.com> parents: 77942 diff changeset	236 'the -b/--browser or -c/--complete options')
77899 da4dd603030b Add syntax highlighter tool Raymond Hettinger <python@rcn.com> parents: diff changeset	237
77928 4d320cd4c6fe Minor cleanups Raymond Hettinger <python@rcn.com> parents: 77920 diff changeset	238 sourcefile = args.sourcefile
77899 da4dd603030b Add syntax highlighter tool Raymond Hettinger <python@rcn.com> parents: diff changeset	239 with open(sourcefile) as f:
77942 1401ff261ced Add examples to the module docstring Raymond Hettinger <python@rcn.com> parents: 77941 diff changeset	240 source = f.read()
78014 77ca62751877 Minor algorithmic tweaks, better factoring, improved docstrings and variable names Raymond Hettinger <python@rcn.com> parents: 77944 diff changeset	241 classified_text = analyze_python(source)
77940 980de5b47d0d Add support for ANSI escape sequences Raymond Hettinger <python@rcn.com> parents: 77938 diff changeset	242
78049 08bfd88027bb Add simple LaTeX markup Raymond Hettinger <python@rcn.com> parents: 78021 diff changeset	243 if args.raw:
08bfd88027bb Add simple LaTeX markup Raymond Hettinger <python@rcn.com> parents: 78021 diff changeset	244 encoded = raw_highlight(classified_text)
08bfd88027bb Add simple LaTeX markup Raymond Hettinger <python@rcn.com> parents: 78021 diff changeset	245 elif args.complete or args.browser:
78014 77ca62751877 Minor algorithmic tweaks, better factoring, improved docstrings and variable names Raymond Hettinger <python@rcn.com> parents: 77944 diff changeset	246 encoded = build_html_page(classified_text, title=sourcefile)
77940 980de5b47d0d Add support for ANSI escape sequences Raymond Hettinger <python@rcn.com> parents: 77938 diff changeset	247 elif args.section:
78014 77ca62751877 Minor algorithmic tweaks, better factoring, improved docstrings and variable names Raymond Hettinger <python@rcn.com> parents: 77944 diff changeset	248 encoded = html_highlight(classified_text)
78049 08bfd88027bb Add simple LaTeX markup Raymond Hettinger <python@rcn.com> parents: 78021 diff changeset	249 elif args.latex:
08bfd88027bb Add simple LaTeX markup Raymond Hettinger <python@rcn.com> parents: 78021 diff changeset	250 encoded = latex_highlight(classified_text, title=sourcefile)
77940 980de5b47d0d Add support for ANSI escape sequences Raymond Hettinger <python@rcn.com> parents: 77938 diff changeset	251 else:
78014 77ca62751877 Minor algorithmic tweaks, better factoring, improved docstrings and variable names Raymond Hettinger <python@rcn.com> parents: 77944 diff changeset	252 encoded = ansi_highlight(classified_text)
77940 980de5b47d0d Add support for ANSI escape sequences Raymond Hettinger <python@rcn.com> parents: 77938 diff changeset	253
77899 da4dd603030b Add syntax highlighter tool Raymond Hettinger <python@rcn.com> parents: diff changeset	254 if args.browser:
da4dd603030b Add syntax highlighter tool Raymond Hettinger <python@rcn.com> parents: diff changeset	255 htmlfile = os.path.splitext(os.path.basename(sourcefile))[0] + '.html'
da4dd603030b Add syntax highlighter tool Raymond Hettinger <python@rcn.com> parents: diff changeset	256 with open(htmlfile, 'w') as f:
77940 980de5b47d0d Add support for ANSI escape sequences Raymond Hettinger <python@rcn.com> parents: 77938 diff changeset	257 f.write(encoded)
77899 da4dd603030b Add syntax highlighter tool Raymond Hettinger <python@rcn.com> parents: diff changeset	258 webbrowser.open('file://' + os.path.abspath(htmlfile))
da4dd603030b Add syntax highlighter tool Raymond Hettinger <python@rcn.com> parents: diff changeset	259 else:
77940 980de5b47d0d Add support for ANSI escape sequences Raymond Hettinger <python@rcn.com> parents: 77938 diff changeset	260 sys.stdout.write(encoded)

Mercurial > cpython-lru_cache-weakref

annotate Tools/scripts/highlight.py @ 84007:dbfbdf2b5c19 3.3