cpython-lru_cache-weakref: Tools/pybench/pybench.py annotate

annotate Tools/pybench/pybench.py @ 78069:a8aa77458e12 3.2

Add c_size_t and c_ssize_t to the ctypes summary table.

author	Antoine Pitrou <solipsis@pitrou.net>
date	Thu, 12 Jul 2012 20:31:50 +0200
parents	66ef5e844e6c
children	f39b26ca7f3d

rev	line source
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	1 #!/usr/local/bin/python -O
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	2
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	3 """ A Python Benchmark Suite
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	4
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	5 """
51443 901cb207b1d3 Merged revisions 69411 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 48555 diff changeset	6 # Note: Please keep this module compatible to Python 2.6.
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	7 #
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	8 # Tests may include features in later Python versions, but these
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	9 # should then be embedded in try-except clauses in the configuration
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	10 # module Setup.py.
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	11 #
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	12
51443 901cb207b1d3 Merged revisions 69411 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 48555 diff changeset	13 from __future__ import print_function
901cb207b1d3 Merged revisions 69411 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 48555 diff changeset	14
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	15 # pybench Copyright
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	16 __copyright__ = """\
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	17 Copyright (c), 1997-2006, Marc-Andre Lemburg (mal@lemburg.com)
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	18 Copyright (c), 2000-2006, eGenix.com Software GmbH (info@egenix.com)
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	19
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	20 All Rights Reserved.
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	21
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	22 Permission to use, copy, modify, and distribute this software and its
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	23 documentation for any purpose and without fee or royalty is hereby
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	24 granted, provided that the above copyright notice appear in all copies
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	25 and that both that copyright notice and this permission notice appear
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	26 in supporting documentation or portions thereof, including
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	27 modifications, that you make.
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	28
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	29 THE AUTHOR MARC-ANDRE LEMBURG DISCLAIMS ALL WARRANTIES WITH REGARD TO
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	30 THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	31 FITNESS, IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL,
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	32 INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	33 FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT,
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	34 NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	35 WITH THE USE OR PERFORMANCE OF THIS SOFTWARE !
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	36 """
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	37
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	38 import sys, time, operator, platform
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	39 from CommandLine import *
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	40
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	41 try:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	42 import cPickle
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	43 pickle = cPickle
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	44 except ImportError:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	45 import pickle
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	46
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	47 # Version number; version history: see README file !
51443 901cb207b1d3 Merged revisions 69411 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 48555 diff changeset	48 __version__ = '2.1'
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	49
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	50 ### Constants
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	51
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	52 # Second fractions
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	53 MILLI_SECONDS = 1e3
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	54 MICRO_SECONDS = 1e6
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	55
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	56 # Percent unit
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	57 PERCENT = 100
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	58
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	59 # Horizontal line length
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	60 LINE = 79
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	61
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	62 # Minimum test run-time
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	63 MIN_TEST_RUNTIME = 1e-3
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	64
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	65 # Number of calibration runs to use for calibrating the tests
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	66 CALIBRATION_RUNS = 20
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	67
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	68 # Number of calibration loops to run for each calibration run
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	69 CALIBRATION_LOOPS = 20
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	70
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	71 # Allow skipping calibration ?
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	72 ALLOW_SKIPPING_CALIBRATION = 1
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	73
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	74 # Timer types
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	75 TIMER_TIME_TIME = 'time.time'
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	76 TIMER_TIME_CLOCK = 'time.clock'
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	77 TIMER_SYSTIMES_PROCESSTIME = 'systimes.processtime'
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	78
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	79 # Choose platform default timer
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	80 if sys.platform[:3] == 'win':
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	81 # On WinXP this has 2.5ms resolution
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	82 TIMER_PLATFORM_DEFAULT = TIMER_TIME_CLOCK
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	83 else:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	84 # On Linux this has 1ms resolution
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	85 TIMER_PLATFORM_DEFAULT = TIMER_TIME_TIME
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	86
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	87 # Print debug information ?
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	88 _debug = 0
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	89
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	90 ### Helpers
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	91
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	92 def get_timer(timertype):
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	93
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	94 if timertype == TIMER_TIME_TIME:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	95 return time.time
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	96 elif timertype == TIMER_TIME_CLOCK:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	97 return time.clock
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	98 elif timertype == TIMER_SYSTIMES_PROCESSTIME:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	99 import systimes
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	100 return systimes.processtime
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	101 else:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	102 raise TypeError('unknown timer type: %s' % timertype)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	103
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	104 def get_machine_details():
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	105
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	106 if _debug:
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	107 print('Getting machine details...')
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	108 buildno, builddate = platform.python_build()
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	109 python = platform.python_version()
48555 739bb0728efe #3092: fix unicode size detection in pybench Antoine Pitrou <solipsis@pitrou.net> parents: 47103 diff changeset	110 if sys.maxunicode == 65535:
40643 47f9efdc0855 Merged revisions 53304-53433,53435-53450 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40587 diff changeset	111 # UCS2 build (standard)
47103 63741c7521aa More 2to3 fixes in the Tools directory. Fixes #2893. Georg Brandl <georg@python.org> parents: 46664 diff changeset	112 unitype = 'UCS2'
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	113 else:
40643 47f9efdc0855 Merged revisions 53304-53433,53435-53450 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40587 diff changeset	114 # UCS4 build (most recent Linux distros)
47103 63741c7521aa More 2to3 fixes in the Tools directory. Fixes #2893. Georg Brandl <georg@python.org> parents: 46664 diff changeset	115 unitype = 'UCS4'
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	116 bits, linkage = platform.architecture()
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	117 return {
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	118 'platform': platform.platform(),
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	119 'processor': platform.processor(),
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	120 'executable': sys.executable,
45773 832c820e9d14 Merged revisions 61239-61249,61252-61257,61260-61264,61269-61275,61278-61279,61285-61286,61288-61290,61298,61303-61305,61312-61314,61317,61329,61332,61344,61350-61351,61363-61376,61378-61379,61382-61383,61387-61388,61392,61395-61396,61402-61403 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 44167 diff changeset	121 'implementation': getattr(platform, 'python_implementation',
832c820e9d14 Merged revisions 61239-61249,61252-61257,61260-61264,61269-61275,61278-61279,61285-61286,61288-61290,61298,61303-61305,61312-61314,61317,61329,61332,61344,61350-61351,61363-61376,61378-61379,61382-61383,61387-61388,61392,61395-61396,61402-61403 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 44167 diff changeset	122 lambda:'n/a')(),
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	123 'python': platform.python_version(),
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	124 'compiler': platform.python_compiler(),
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	125 'buildno': buildno,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	126 'builddate': builddate,
47103 63741c7521aa More 2to3 fixes in the Tools directory. Fixes #2893. Georg Brandl <georg@python.org> parents: 46664 diff changeset	127 'unicode': unitype,
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	128 'bits': bits,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	129 }
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	130
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	131 def print_machine_details(d, indent=''):
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	132
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	133 l = ['Machine Details:',
40643 47f9efdc0855 Merged revisions 53304-53433,53435-53450 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40587 diff changeset	134 ' Platform ID: %s' % d.get('platform', 'n/a'),
47f9efdc0855 Merged revisions 53304-53433,53435-53450 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40587 diff changeset	135 ' Processor: %s' % d.get('processor', 'n/a'),
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	136 '',
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	137 'Python:',
40643 47f9efdc0855 Merged revisions 53304-53433,53435-53450 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40587 diff changeset	138 ' Implementation: %s' % d.get('implementation', 'n/a'),
47f9efdc0855 Merged revisions 53304-53433,53435-53450 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40587 diff changeset	139 ' Executable: %s' % d.get('executable', 'n/a'),
47f9efdc0855 Merged revisions 53304-53433,53435-53450 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40587 diff changeset	140 ' Version: %s' % d.get('python', 'n/a'),
47f9efdc0855 Merged revisions 53304-53433,53435-53450 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40587 diff changeset	141 ' Compiler: %s' % d.get('compiler', 'n/a'),
47f9efdc0855 Merged revisions 53304-53433,53435-53450 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40587 diff changeset	142 ' Bits: %s' % d.get('bits', 'n/a'),
47f9efdc0855 Merged revisions 53304-53433,53435-53450 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40587 diff changeset	143 ' Build: %s (#%s)' % (d.get('builddate', 'n/a'),
47f9efdc0855 Merged revisions 53304-53433,53435-53450 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40587 diff changeset	144 d.get('buildno', 'n/a')),
47f9efdc0855 Merged revisions 53304-53433,53435-53450 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40587 diff changeset	145 ' Unicode: %s' % d.get('unicode', 'n/a'),
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	146 ]
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	147 joiner = '\n' + indent
863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	148 print(indent + joiner.join(l) + '\n')
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	149
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	150 ### Test baseclass
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	151
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	152 class Test:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	153
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	154 """ All test must have this class as baseclass. It provides
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	155 the necessary interface to the benchmark machinery.
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	156
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	157 The tests must set .rounds to a value high enough to let the
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	158 test run between 20-50 seconds. This is needed because
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	159 clock()-timing only gives rather inaccurate values (on Linux,
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	160 for example, it is accurate to a few hundreths of a
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	161 second). If you don't want to wait that long, use a warp
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	162 factor larger than 1.
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	163
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	164 It is also important to set the .operations variable to a
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	165 value representing the number of "virtual operations" done per
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	166 call of .run().
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	167
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	168 If you change a test in some way, don't forget to increase
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 40643 diff changeset	169 its version number.
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	170
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	171 """
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	172
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	173 ### Instance variables that each test should override
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	174
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	175 # Version number of the test as float (x.yy); this is important
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	176 # for comparisons of benchmark runs - tests with unequal version
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	177 # number will not get compared.
51443 901cb207b1d3 Merged revisions 69411 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 48555 diff changeset	178 version = 2.1
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 37879 diff changeset	179
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	180 # The number of abstract operations done in each round of the
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	181 # test. An operation is the basic unit of what you want to
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	182 # measure. The benchmark will output the amount of run-time per
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	183 # operation. Note that in order to raise the measured timings
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	184 # significantly above noise level, it is often required to repeat
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	185 # sets of operations more than once per test round. The measured
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	186 # overhead per test round should be less than 1 second.
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	187 operations = 1
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	188
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	189 # Number of rounds to execute per test run. This should be
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	190 # adjusted to a figure that results in a test run-time of between
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	191 # 1-2 seconds.
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	192 rounds = 100000
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	193
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	194 ### Internal variables
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	195
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	196 # Mark this class as implementing a test
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	197 is_a_test = 1
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	198
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	199 # Last timing: (real, run, overhead)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	200 last_timing = (0.0, 0.0, 0.0)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	201
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	202 # Warp factor to use for this test
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	203 warp = 1
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	204
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	205 # Number of calibration runs to use
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	206 calibration_runs = CALIBRATION_RUNS
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	207
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	208 # List of calibration timings
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	209 overhead_times = None
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	210
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	211 # List of test run timings
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	212 times = []
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	213
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	214 # Timer used for the benchmark
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	215 timer = TIMER_PLATFORM_DEFAULT
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	216
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	217 def __init__(self, warp=None, calibration_runs=None, timer=None):
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	218
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	219 # Set parameters
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	220 if warp is not None:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	221 self.rounds = int(self.rounds / warp)
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 37879 diff changeset	222 if self.rounds == 0:
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	223 raise ValueError('warp factor set too high')
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	224 self.warp = warp
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	225 if calibration_runs is not None:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	226 if (not ALLOW_SKIPPING_CALIBRATION and
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	227 calibration_runs < 1):
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	228 raise ValueError('at least one calibration run is required')
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	229 self.calibration_runs = calibration_runs
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	230 if timer is not None:
57450 0db67153d25f Merged revisions 75264,75268,75293,75318,75391-75392,75436,75478,75971,76003,76058,76140-76141,76231,76380,76428-76429 via svnmerge from Benjamin Peterson <benjamin@python.org> parents: 51443 diff changeset	231 self.timer = timer
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	232
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	233 # Init variables
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	234 self.times = []
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	235 self.overhead_times = []
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	236
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	237 # We want these to be in the instance dict, so that pickle
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	238 # saves them
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	239 self.version = self.version
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	240 self.operations = self.operations
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	241 self.rounds = self.rounds
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	242
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	243 def get_timer(self):
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	244
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	245 """ Return the timer function to use for the test.
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	246
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	247 """
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	248 return get_timer(self.timer)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	249
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	250 def compatible(self, other):
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	251
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	252 """ Return 1/0 depending on whether the test is compatible
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	253 with the other Test instance or not.
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	254
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	255 """
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	256 if self.version != other.version:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	257 return 0
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	258 if self.rounds != other.rounds:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	259 return 0
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	260 return 1
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	261
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	262 def calibrate_test(self):
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	263
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	264 if self.calibration_runs == 0:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	265 self.overhead_times = [0.0]
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	266 return
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	267
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	268 calibrate = self.calibrate
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	269 timer = self.get_timer()
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	270 calibration_loops = range(CALIBRATION_LOOPS)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	271
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	272 # Time the calibration loop overhead
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	273 prep_times = []
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	274 for i in range(self.calibration_runs):
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	275 t = timer()
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	276 for i in calibration_loops:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	277 pass
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	278 t = timer() - t
69546 7569870a8236 pybench prep_times calculation error (closes #11895) Jesus Cea <jcea@jcea.es> parents: 68488 diff changeset	279 prep_times.append(t / CALIBRATION_LOOPS)
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	280 min_prep_time = min(prep_times)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	281 if _debug:
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	282 print()
863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	283 print('Calib. prep time = %.6fms' % (
863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	284 min_prep_time * MILLI_SECONDS))
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	285
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	286 # Time the calibration runs (doing CALIBRATION_LOOPS loops of
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	287 # .calibrate() method calls each)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	288 for i in range(self.calibration_runs):
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	289 t = timer()
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	290 for i in calibration_loops:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	291 calibrate()
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	292 t = timer() - t
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	293 self.overhead_times.append(t / CALIBRATION_LOOPS
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	294 - min_prep_time)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	295
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	296 # Check the measured times
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	297 min_overhead = min(self.overhead_times)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	298 max_overhead = max(self.overhead_times)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	299 if _debug:
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	300 print('Calib. overhead time = %.6fms' % (
863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	301 min_overhead * MILLI_SECONDS))
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	302 if min_overhead < 0.0:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	303 raise ValueError('calibration setup did not work')
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	304 if max_overhead - min_overhead > 0.1:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	305 raise ValueError(
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	306 'overhead calibration timing range too inaccurate: '
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	307 '%r - %r' % (min_overhead, max_overhead))
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	308
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	309 def run(self):
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	310
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	311 """ Run the test in two phases: first calibrate, then
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	312 do the actual test. Be careful to keep the calibration
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	313 timing low w/r to the test timing.
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 37879 diff changeset	314
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	315 """
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	316 test = self.test
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	317 timer = self.get_timer()
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	318
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	319 # Get calibration
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	320 min_overhead = min(self.overhead_times)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	321
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	322 # Test run
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	323 t = timer()
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	324 test()
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	325 t = timer() - t
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	326 if t < MIN_TEST_RUNTIME:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	327 raise ValueError('warp factor too high: '
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	328 'test times are < 10ms')
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	329 eff_time = t - min_overhead
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	330 if eff_time < 0:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	331 raise ValueError('wrong calibration')
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	332 self.last_timing = (eff_time, t, min_overhead)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	333 self.times.append(eff_time)
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	334
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	335 def calibrate(self):
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	336
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 37879 diff changeset	337 """ Calibrate the test.
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	338
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 37879 diff changeset	339 This method should execute everything that is needed to
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 37879 diff changeset	340 setup and run the test - except for the actual operations
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 37879 diff changeset	341 that you intend to measure. pybench uses this method to
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 37879 diff changeset	342 measure the test implementation overhead.
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	343
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	344 """
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	345 return
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	346
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	347 def test(self):
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	348
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 37879 diff changeset	349 """ Run the test.
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	350
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 37879 diff changeset	351 The test needs to run self.rounds executing
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 37879 diff changeset	352 self.operations number of operations each.
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	353
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	354 """
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	355 return
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 37879 diff changeset	356
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	357 def stat(self):
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	358
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	359 """ Return test run statistics as tuple:
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 37879 diff changeset	360
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	361 (minimum run time,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	362 average run time,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	363 total run time,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	364 average time per operation,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	365 minimum overhead time)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	366
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	367 """
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	368 runs = len(self.times)
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	369 if runs == 0:
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	370 return 0.0, 0.0, 0.0, 0.0
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	371 min_time = min(self.times)
39627 e86c0b29b4a2 Kill reduce(). A coproduction of John Reese, Jacques Frechet, and Alex M. Guido van Rossum <guido@python.org> parents: 39442 diff changeset	372 total_time = sum(self.times)
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	373 avg_time = total_time / float(runs)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	374 operation_avg = total_time / float(runs
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	375 * self.rounds
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	376 * self.operations)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	377 if self.overhead_times:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	378 min_overhead = min(self.overhead_times)
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	379 else:
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	380 min_overhead = self.last_timing[2]
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	381 return min_time, avg_time, total_time, operation_avg, min_overhead
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	382
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	383 ### Load Setup
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	384
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	385 # This has to be done after the definition of the Test class, since
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	386 # the Setup module will import subclasses using this class.
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	387
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	388 import Setup
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	389
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	390 ### Benchmark base class
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	391
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	392 class Benchmark:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	393
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	394 # Name of the benchmark
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	395 name = ''
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	396
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	397 # Number of benchmark rounds to run
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	398 rounds = 1
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	399
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	400 # Warp factor use to run the tests
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	401 warp = 1 # Warp factor
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	402
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	403 # Average benchmark round time
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	404 roundtime = 0
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	405
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	406 # Benchmark version number as float x.yy
51443 901cb207b1d3 Merged revisions 69411 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 48555 diff changeset	407 version = 2.1
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	408
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	409 # Produce verbose output ?
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	410 verbose = 0
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	411
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	412 # Dictionary with the machine details
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	413 machine_details = None
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	414
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	415 # Timer used for the benchmark
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	416 timer = TIMER_PLATFORM_DEFAULT
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	417
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	418 def __init__(self, name, verbose=None, timer=None, warp=None,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	419 calibration_runs=None):
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	420
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	421 if name:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	422 self.name = name
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	423 else:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	424 self.name = '%04i-%02i-%02i %02i:%02i:%02i' % \
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	425 (time.localtime(time.time())[:6])
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	426 if verbose is not None:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	427 self.verbose = verbose
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	428 if timer is not None:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	429 self.timer = timer
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	430 if warp is not None:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	431 self.warp = warp
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	432 if calibration_runs is not None:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	433 self.calibration_runs = calibration_runs
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	434
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	435 # Init vars
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	436 self.tests = {}
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	437 if _debug:
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	438 print('Getting machine details...')
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	439 self.machine_details = get_machine_details()
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	440
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	441 # Make .version an instance attribute to have it saved in the
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	442 # Benchmark pickle
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	443 self.version = self.version
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	444
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	445 def get_timer(self):
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	446
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	447 """ Return the timer function to use for the test.
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	448
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	449 """
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	450 return get_timer(self.timer)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	451
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	452 def compatible(self, other):
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	453
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	454 """ Return 1/0 depending on whether the benchmark is
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	455 compatible with the other Benchmark instance or not.
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	456
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	457 """
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	458 if self.version != other.version:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	459 return 0
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	460 if (self.machine_details == other.machine_details and
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	461 self.timer != other.timer):
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	462 return 0
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	463 if (self.calibration_runs == 0 and
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	464 other.calibration_runs != 0):
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	465 return 0
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	466 if (self.calibration_runs != 0 and
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	467 other.calibration_runs == 0):
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	468 return 0
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	469 return 1
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	470
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	471 def load_tests(self, setupmod, limitnames=None):
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	472
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	473 # Add tests
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	474 if self.verbose:
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	475 print('Searching for tests ...')
863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	476 print('--------------------------------------')
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	477 for testclass in setupmod.__dict__.values():
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	478 if not hasattr(testclass, 'is_a_test'):
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 37879 diff changeset	479 continue
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	480 name = testclass.__name__
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 37879 diff changeset	481 if name == 'Test':
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 37879 diff changeset	482 continue
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	483 if (limitnames is not None and
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	484 limitnames.search(name) is None):
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 37879 diff changeset	485 continue
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	486 self.tests[name] = testclass(
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	487 warp=self.warp,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	488 calibration_runs=self.calibration_runs,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	489 timer=self.timer)
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	490 l = sorted(self.tests)
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	491 if self.verbose:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	492 for name in l:
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	493 print(' %s' % name)
863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	494 print('--------------------------------------')
863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	495 print(' %i tests found' % len(l))
863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	496 print()
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	497
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	498 def calibrate(self):
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	499
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	500 print('Calibrating tests. Please wait...', end=' ')
46664 031dfe455585 Merged revisions 62425-62429,62434-62436,62441,62444,62446-62448,62450-62455,62463,62465-62466,62469,62474,62476-62478,62480,62485,62492,62497-62498,62500,62507,62513-62514,62516,62521,62531,62535,62545-62546,62548-62551,62553-62559,62569,62574,62577,62593,62595,62604-62606,62608,62616,62626-62627,62636,62638,62644-62645,62647-62648,62651-62653,62656,62661,62663,62680,62686-62687,62696,62699-62703,62711 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 45773 diff changeset	501 sys.stdout.flush()
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	502 if self.verbose:
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	503 print()
863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	504 print()
863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	505 print('Test min max')
863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	506 print('-' * LINE)
863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	507 tests = sorted(self.tests.items())
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	508 for i in range(len(tests)):
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	509 name, test = tests[i]
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	510 test.calibrate_test()
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	511 if self.verbose:
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	512 print('%30s: %6.3fms %6.3fms' % \
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	513 (name,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	514 min(test.overhead_times) * MILLI_SECONDS,
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	515 max(test.overhead_times) * MILLI_SECONDS))
40643 47f9efdc0855 Merged revisions 53304-53433,53435-53450 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40587 diff changeset	516 if self.verbose:
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	517 print()
863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	518 print('Done with the calibration.')
40643 47f9efdc0855 Merged revisions 53304-53433,53435-53450 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40587 diff changeset	519 else:
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	520 print('done.')
863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	521 print()
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	522
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	523 def run(self):
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	524
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	525 tests = sorted(self.tests.items())
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	526 timer = self.get_timer()
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	527 print('Running %i round(s) of the suite at warp factor %i:' % \
863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	528 (self.rounds, self.warp))
863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	529 print()
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	530 self.roundtimes = []
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	531 for i in range(self.rounds):
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	532 if self.verbose:
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	533 print(' Round %-25i effective absolute overhead' % (i+1))
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	534 total_eff_time = 0.0
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	535 for j in range(len(tests)):
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	536 name, test = tests[j]
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	537 if self.verbose:
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	538 print('%30s:' % name, end=' ')
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	539 test.run()
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	540 (eff_time, abs_time, min_overhead) = test.last_timing
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	541 total_eff_time = total_eff_time + eff_time
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	542 if self.verbose:
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	543 print(' %5.0fms %5.0fms %7.3fms' % \
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	544 (eff_time * MILLI_SECONDS,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	545 abs_time * MILLI_SECONDS,
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	546 min_overhead * MILLI_SECONDS))
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	547 self.roundtimes.append(total_eff_time)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	548 if self.verbose:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42132 diff changeset	549 print(' '
46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42132 diff changeset	550 ' ------------------------------')
46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42132 diff changeset	551 print(' '
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	552 ' Totals: %6.0fms' %
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42132 diff changeset	553 (total_eff_time * MILLI_SECONDS))
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	554 print()
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 37879 diff changeset	555 else:
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	556 print('* Round %i done in %.3f seconds.' % (i+1,
863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	557 total_eff_time))
863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	558 print()
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	559
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	560 def stat(self):
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	561
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	562 """ Return benchmark run statistics as tuple:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	563
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	564 (minimum round time,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	565 average round time,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	566 maximum round time)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	567
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	568 XXX Currently not used, since the benchmark does test
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	569 statistics across all rounds.
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	570
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	571 """
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	572 runs = len(self.roundtimes)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	573 if runs == 0:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	574 return 0.0, 0.0
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	575 min_time = min(self.roundtimes)
39627 e86c0b29b4a2 Kill reduce(). A coproduction of John Reese, Jacques Frechet, and Alex M. Guido van Rossum <guido@python.org> parents: 39442 diff changeset	576 total_time = sum(self.roundtimes)
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	577 avg_time = total_time / float(runs)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	578 max_time = max(self.roundtimes)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	579 return (min_time, avg_time, max_time)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	580
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	581 def print_header(self, title='Benchmark'):
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	582
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	583 print('-' * LINE)
863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	584 print('%s: %s' % (title, self.name))
863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	585 print('-' * LINE)
863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	586 print()
863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	587 print(' Rounds: %s' % self.rounds)
863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	588 print(' Warp: %s' % self.warp)
863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	589 print(' Timer: %s' % self.timer)
863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	590 print()
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	591 if self.machine_details:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	592 print_machine_details(self.machine_details, indent=' ')
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	593 print()
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	594
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	595 def print_benchmark(self, hidenoise=0, limitnames=None):
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	596
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42132 diff changeset	597 print('Test '
46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42132 diff changeset	598 ' minimum average operation overhead')
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	599 print('-' * LINE)
863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	600 tests = sorted(self.tests.items())
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	601 total_min_time = 0.0
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	602 total_avg_time = 0.0
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	603 for name, test in tests:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	604 if (limitnames is not None and
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	605 limitnames.search(name) is None):
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	606 continue
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	607 (min_time,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	608 avg_time,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	609 total_time,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	610 op_avg,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	611 min_overhead) = test.stat()
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	612 total_min_time = total_min_time + min_time
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	613 total_avg_time = total_avg_time + avg_time
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	614 print('%30s: %5.0fms %5.0fms %6.2fus %7.3fms' % \
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	615 (name,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	616 min_time * MILLI_SECONDS,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	617 avg_time * MILLI_SECONDS,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	618 op_avg * MICRO_SECONDS,
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	619 min_overhead *MILLI_SECONDS))
863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	620 print('-' * LINE)
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42132 diff changeset	621 print('Totals: '
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	622 ' %6.0fms %6.0fms' %
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	623 (total_min_time * MILLI_SECONDS,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	624 total_avg_time * MILLI_SECONDS,
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42132 diff changeset	625 ))
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	626 print()
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 37879 diff changeset	627
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	628 def print_comparison(self, compare_to, hidenoise=0, limitnames=None):
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	629
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	630 # Check benchmark versions
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	631 if compare_to.version != self.version:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42132 diff changeset	632 print('* Benchmark versions differ: '
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	633 'cannot compare this benchmark to "%s" !' %
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42132 diff changeset	634 compare_to.name)
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	635 print()
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	636 self.print_benchmark(hidenoise=hidenoise,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	637 limitnames=limitnames)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	638 return
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	639
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	640 # Print header
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	641 compare_to.print_header('Comparing with')
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42132 diff changeset	642 print('Test '
46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42132 diff changeset	643 ' minimum run-time average run-time')
46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42132 diff changeset	644 print(' '
46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42132 diff changeset	645 ' this other diff this other diff')
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	646 print('-' * LINE)
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	647
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	648 # Print test comparisons
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	649 tests = sorted(self.tests.items())
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	650 total_min_time = other_total_min_time = 0.0
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	651 total_avg_time = other_total_avg_time = 0.0
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	652 benchmarks_compatible = self.compatible(compare_to)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	653 tests_compatible = 1
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	654 for name, test in tests:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	655 if (limitnames is not None and
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	656 limitnames.search(name) is None):
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	657 continue
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	658 (min_time,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	659 avg_time,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	660 total_time,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	661 op_avg,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	662 min_overhead) = test.stat()
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	663 total_min_time = total_min_time + min_time
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	664 total_avg_time = total_avg_time + avg_time
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	665 try:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	666 other = compare_to.tests[name]
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	667 except KeyError:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	668 other = None
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	669 if other is None:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	670 # Other benchmark doesn't include the given test
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	671 min_diff, avg_diff = 'n/a', 'n/a'
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	672 other_min_time = 0.0
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	673 other_avg_time = 0.0
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	674 tests_compatible = 0
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	675 else:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	676 (other_min_time,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	677 other_avg_time,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	678 other_total_time,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	679 other_op_avg,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	680 other_min_overhead) = other.stat()
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	681 other_total_min_time = other_total_min_time + other_min_time
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	682 other_total_avg_time = other_total_avg_time + other_avg_time
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	683 if (benchmarks_compatible and
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	684 test.compatible(other)):
68488 0991b40e895d #11515: fix several typos. Patch by Piotr Kasprzyk. Ezio Melotti <ezio.melotti@gmail.com> parents: 51443 diff changeset	685 # Both benchmark and tests are comparable
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	686 min_diff = ((min_time * self.warp) /
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	687 (other_min_time * other.warp) - 1.0)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	688 avg_diff = ((avg_time * self.warp) /
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	689 (other_avg_time * other.warp) - 1.0)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	690 if hidenoise and abs(min_diff) < 10.0:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	691 min_diff = ''
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	692 else:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	693 min_diff = '%+5.1f%%' % (min_diff * PERCENT)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	694 if hidenoise and abs(avg_diff) < 10.0:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	695 avg_diff = ''
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	696 else:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	697 avg_diff = '%+5.1f%%' % (avg_diff * PERCENT)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	698 else:
68488 0991b40e895d #11515: fix several typos. Patch by Piotr Kasprzyk. Ezio Melotti <ezio.melotti@gmail.com> parents: 51443 diff changeset	699 # Benchmark or tests are not comparable
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	700 min_diff, avg_diff = 'n/a', 'n/a'
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	701 tests_compatible = 0
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	702 print('%30s: %5.0fms %5.0fms %7s %5.0fms %5.0fms %7s' % \
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	703 (name,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	704 min_time * MILLI_SECONDS,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	705 other_min_time * MILLI_SECONDS * compare_to.warp / self.warp,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	706 min_diff,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	707 avg_time * MILLI_SECONDS,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	708 other_avg_time * MILLI_SECONDS * compare_to.warp / self.warp,
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	709 avg_diff))
863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	710 print('-' * LINE)
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	711
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	712 # Summarise test results
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	713 if not benchmarks_compatible or not tests_compatible:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	714 min_diff, avg_diff = 'n/a', 'n/a'
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	715 else:
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	716 if other_total_min_time != 0.0:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	717 min_diff = '%+5.1f%%' % (
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	718 ((total_min_time * self.warp) /
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	719 (other_total_min_time * compare_to.warp) - 1.0) * PERCENT)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	720 else:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	721 min_diff = 'n/a'
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	722 if other_total_avg_time != 0.0:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	723 avg_diff = '%+5.1f%%' % (
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	724 ((total_avg_time * self.warp) /
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	725 (other_total_avg_time * compare_to.warp) - 1.0) * PERCENT)
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	726 else:
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	727 avg_diff = 'n/a'
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42132 diff changeset	728 print('Totals: '
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	729 ' %5.0fms %5.0fms %7s %5.0fms %5.0fms %7s' %
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	730 (total_min_time * MILLI_SECONDS,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	731 (other_total_min_time * compare_to.warp/self.warp
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	732 * MILLI_SECONDS),
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	733 min_diff,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	734 total_avg_time * MILLI_SECONDS,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	735 (other_total_avg_time * compare_to.warp/self.warp
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	736 * MILLI_SECONDS),
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	737 avg_diff
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42132 diff changeset	738 ))
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	739 print()
863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	740 print('(this=%s, other=%s)' % (self.name,
863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	741 compare_to.name))
863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	742 print()
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	743
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	744 class PyBenchCmdline(Application):
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	745
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	746 header = ("PYBENCH - a benchmark test suite for Python "
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	747 "interpreters/compilers.")
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	748
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	749 version = __version__
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	750
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	751 debug = _debug
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	752
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	753 options = [ArgumentOption('-n',
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	754 'number of rounds',
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	755 Setup.Number_of_rounds),
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	756 ArgumentOption('-f',
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	757 'save benchmark to file arg',
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	758 ''),
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	759 ArgumentOption('-c',
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	760 'compare benchmark with the one in file arg',
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	761 ''),
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	762 ArgumentOption('-s',
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	763 'show benchmark in file arg, then exit',
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	764 ''),
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	765 ArgumentOption('-w',
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	766 'set warp factor to arg',
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	767 Setup.Warp_factor),
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	768 ArgumentOption('-t',
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	769 'run only tests with names matching arg',
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	770 ''),
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	771 ArgumentOption('-C',
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	772 'set the number of calibration runs to arg',
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	773 CALIBRATION_RUNS),
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	774 SwitchOption('-d',
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	775 'hide noise in comparisons',
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	776 0),
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	777 SwitchOption('-v',
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	778 'verbose output (not recommended)',
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	779 0),
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	780 SwitchOption('--with-gc',
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	781 'enable garbage collection',
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	782 0),
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	783 SwitchOption('--with-syscheck',
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	784 'use default sys check interval',
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	785 0),
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	786 ArgumentOption('--timer',
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	787 'use given timer',
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	788 TIMER_PLATFORM_DEFAULT),
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	789 ]
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	790
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	791 about = """\
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	792 The normal operation is to run the suite and display the
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	793 results. Use -f to save them for later reuse or comparisons.
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	794
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	795 Available timers:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	796
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	797 time.time
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	798 time.clock
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	799 systimes.processtime
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	800
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	801 Examples:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	802
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	803 python2.1 pybench.py -f p21.pybench
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	804 python2.5 pybench.py -f p25.pybench
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	805 python pybench.py -s p25.pybench -c p21.pybench
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	806 """
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	807 copyright = __copyright__
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	808
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	809 def main(self):
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	810
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	811 rounds = self.values['-n']
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	812 reportfile = self.values['-f']
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	813 show_bench = self.values['-s']
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	814 compare_to = self.values['-c']
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	815 hidenoise = self.values['-d']
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	816 warp = int(self.values['-w'])
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	817 withgc = self.values['--with-gc']
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 37879 diff changeset	818 limitnames = self.values['-t']
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	819 if limitnames:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	820 if _debug:
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	821 print('* limiting test names to one with substring "%s"' % \
863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	822 limitnames)
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	823 limitnames = re.compile(limitnames, re.I)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	824 else:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	825 limitnames = None
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 37879 diff changeset	826 verbose = self.verbose
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	827 withsyscheck = self.values['--with-syscheck']
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	828 calibration_runs = self.values['-C']
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	829 timer = self.values['--timer']
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 37879 diff changeset	830
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	831 print('-' * LINE)
863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	832 print('PYBENCH %s' % __version__)
863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	833 print('-' * LINE)
863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	834 print('* using %s %s' % (
45773 832c820e9d14 Merged revisions 61239-61249,61252-61257,61260-61264,61269-61275,61278-61279,61285-61286,61288-61290,61298,61303-61305,61312-61314,61317,61329,61332,61344,61350-61351,61363-61376,61378-61379,61382-61383,61387-61388,61392,61395-61396,61402-61403 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 44167 diff changeset	835 getattr(platform, 'python_implementation', lambda:'Python')(),
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	836 ' '.join(sys.version.split())))
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 37879 diff changeset	837
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	838 # Switch off garbage collection
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	839 if not withgc:
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	840 try:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	841 import gc
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	842 except ImportError:
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	843 print('* Python version doesn\'t support garbage collection')
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	844 else:
40643 47f9efdc0855 Merged revisions 53304-53433,53435-53450 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40587 diff changeset	845 try:
47f9efdc0855 Merged revisions 53304-53433,53435-53450 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40587 diff changeset	846 gc.disable()
47f9efdc0855 Merged revisions 53304-53433,53435-53450 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40587 diff changeset	847 except NotImplementedError:
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	848 print('* Python version doesn\'t support gc.disable')
40643 47f9efdc0855 Merged revisions 53304-53433,53435-53450 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40587 diff changeset	849 else:
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	850 print('* disabled garbage collection')
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	851
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	852 # "Disable" sys check interval
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	853 if not withsyscheck:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	854 # Too bad the check interval uses an int instead of a long...
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	855 value = 2147483647
40643 47f9efdc0855 Merged revisions 53304-53433,53435-53450 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40587 diff changeset	856 try:
47f9efdc0855 Merged revisions 53304-53433,53435-53450 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40587 diff changeset	857 sys.setcheckinterval(value)
47f9efdc0855 Merged revisions 53304-53433,53435-53450 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40587 diff changeset	858 except (AttributeError, NotImplementedError):
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	859 print('* Python version doesn\'t support sys.setcheckinterval')
40643 47f9efdc0855 Merged revisions 53304-53433,53435-53450 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40587 diff changeset	860 else:
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	861 print('* system check interval set to maximum: %s' % value)
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	862
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	863 if timer == TIMER_SYSTIMES_PROCESSTIME:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	864 import systimes
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	865 print('* using timer: systimes.processtime (%s)' % \
863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	866 systimes.SYSTIMES_IMPLEMENTATION)
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	867 else:
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	868 print('* using timer: %s' % timer)
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 37879 diff changeset	869
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	870 print()
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	871
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	872 if compare_to:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	873 try:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	874 f = open(compare_to,'rb')
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	875 bench = pickle.load(f)
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	876 bench.name = compare_to
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	877 f.close()
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	878 compare_to = bench
40587 6b18a095c2e7 SF patch 1631942 by Collin Winter: Guido van Rossum <guido@python.org> parents: 40450 diff changeset	879 except IOError as reason:
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	880 print('* Error opening/reading file %s: %s' % (
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	881 repr(compare_to),
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	882 reason))
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 37879 diff changeset	883 compare_to = None
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	884
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	885 if show_bench:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	886 try:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	887 f = open(show_bench,'rb')
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	888 bench = pickle.load(f)
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	889 bench.name = show_bench
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	890 f.close()
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	891 bench.print_header()
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	892 if compare_to:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	893 bench.print_comparison(compare_to,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	894 hidenoise=hidenoise,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	895 limitnames=limitnames)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	896 else:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	897 bench.print_benchmark(hidenoise=hidenoise,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	898 limitnames=limitnames)
40587 6b18a095c2e7 SF patch 1631942 by Collin Winter: Guido van Rossum <guido@python.org> parents: 40450 diff changeset	899 except IOError as reason:
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	900 print('* Error opening/reading file %s: %s' % (
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	901 repr(show_bench),
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	902 reason))
863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	903 print()
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	904 return
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	905
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	906 if reportfile:
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	907 print('Creating benchmark: %s (rounds=%i, warp=%i)' % \
863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	908 (reportfile, rounds, warp))
863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	909 print()
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	910
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	911 # Create benchmark object
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	912 bench = Benchmark(reportfile,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	913 verbose=verbose,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	914 timer=timer,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	915 warp=warp,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	916 calibration_runs=calibration_runs)
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	917 bench.rounds = rounds
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	918 bench.load_tests(Setup, limitnames=limitnames)
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	919 try:
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	920 bench.calibrate()
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	921 bench.run()
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	922 except KeyboardInterrupt:
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	923 print()
863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	924 print('*** KeyboardInterrupt -- Aborting')
863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	925 print()
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	926 return
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	927 bench.print_header()
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	928 if compare_to:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	929 bench.print_comparison(compare_to,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	930 hidenoise=hidenoise,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	931 limitnames=limitnames)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	932 else:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	933 bench.print_benchmark(hidenoise=hidenoise,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	934 limitnames=limitnames)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	935
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	936 # Ring bell
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	937 sys.stderr.write('\007')
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	938
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	939 if reportfile:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	940 try:
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	941 f = open(reportfile,'wb')
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	942 bench.name = reportfile
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	943 pickle.dump(bench,f)
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	944 f.close()
40587 6b18a095c2e7 SF patch 1631942 by Collin Winter: Guido van Rossum <guido@python.org> parents: 40450 diff changeset	945 except IOError as reason:
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	946 print('* Error opening/writing reportfile')
40587 6b18a095c2e7 SF patch 1631942 by Collin Winter: Guido van Rossum <guido@python.org> parents: 40450 diff changeset	947 except IOError as reason:
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	948 print('* Error opening/writing reportfile %s: %s' % (
40450 cc992d75d5b3 Four months of trunk changes (including a few releases...) Thomas Wouters <thomas@python.org> parents: 39627 diff changeset	949 reportfile,
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	950 reason))
863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 41554 diff changeset	951 print()
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	952
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	953 if __name__ == '__main__':
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: diff changeset	954 PyBenchCmdline()

Mercurial > cpython-lru_cache-weakref

annotate Tools/pybench/pybench.py @ 78069:a8aa77458e12 3.2