Mercurial > cpython-lru_cache-weakref

annotate Tools/pybench/README @ 79675:c0a2294de787 2.7

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Issue #16199: Backport 2to3 typo fix from 3.2.
author Chris Jerdonek <chris.jerdonek@gmail.com>
date Thu, 11 Oct 2012 16:15:46 -0700
parents 2167105123e4
children 1ecb712fcf22
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
37857
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
1 ________________________________________________________________________
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
2
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
3 PYBENCH - A Python Benchmark Suite
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
4 ________________________________________________________________________
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
5
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
6 Extendable suite of of low-level benchmarks for measuring
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
7 the performance of the Python implementation
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
8 (interpreter, compiler or VM).
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
9
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
10 pybench is a collection of tests that provides a standardized way to
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
11 measure the performance of Python implementations. It takes a very
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
12 close look at different aspects of Python programs and let's you
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
13 decide which factors are more important to you than others, rather
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
14 than wrapping everything up in one number, like the other performance
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
15 tests do (e.g. pystone which is included in the Python Standard
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
16 Library).
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
17
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
18 pybench has been used in the past by several Python developers to
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
19 track down performance bottlenecks or to demonstrate the impact of
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
20 optimizations and new features in Python.
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
21
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
22 The command line interface for pybench is the file pybench.py. Run
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
23 this script with option '--help' to get a listing of the possible
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
24 options. Without options, pybench will simply execute the benchmark
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
25 and then print out a report to stdout.
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
26
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
27
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
28 Micro-Manual
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
29 ------------
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
30
38801
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
31 Run 'pybench.py -h' to see the help screen. Run 'pybench.py' to run
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
32 the benchmark suite using default settings and 'pybench.py -f <file>'
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
33 to have it store the results in a file too.
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
34
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
35 It is usually a good idea to run pybench.py multiple times to see
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
36 whether the environment, timers and benchmark run-times are suitable
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
37 for doing benchmark tests.
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
38
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
39 You can use the comparison feature of pybench.py ('pybench.py -c
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
40 <file>') to check how well the system behaves in comparison to a
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
41 reference run.
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
42
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
43 If the differences are well below 10% for each test, then you have a
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
44 system that is good for doing benchmark testings. Of you get random
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
45 differences of more than 10% or significant differences between the
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
46 values for minimum and average time, then you likely have some
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
47 background processes running which cause the readings to become
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
48 inconsistent. Examples include: web-browsers, email clients, RSS
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
49 readers, music players, backup programs, etc.
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
50
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
51 If you are only interested in a few tests of the whole suite, you can
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
52 use the filtering option, e.g. 'pybench.py -t string' will only
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
53 run/show the tests that have 'string' in their name.
37857
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
54
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
55 This is the current output of pybench.py --help:
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
56
38801
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
57 """
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
58 ------------------------------------------------------------------------
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
59 PYBENCH - a benchmark test suite for Python interpreters/compilers.
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
60 ------------------------------------------------------------------------
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
61
37857
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
62 Synopsis:
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
63 pybench.py [option] files...
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
64
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
65 Options and default settings:
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
66 -n arg number of rounds (10)
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
67 -f arg save benchmark to file arg ()
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
68 -c arg compare benchmark with the one in file arg ()
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
69 -s arg show benchmark in file arg, then exit ()
38801
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
70 -w arg set warp factor to arg (10)
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
71 -t arg run only tests with names matching arg ()
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
72 -C arg set the number of calibration runs to arg (20)
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
73 -d hide noise in comparisons (0)
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
74 -v verbose output (not recommended) (0)
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
75 --with-gc enable garbage collection (0)
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
76 --with-syscheck use default sys check interval (0)
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
77 --timer arg use given timer (time.time)
37857
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
78 -h show this help text
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
79 --help show this help text
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
80 --debug enable debugging
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
81 --copyright show copyright
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
82 --examples show examples of usage
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
83
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
84 Version:
38801
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
85 2.0
37857
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
86
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
87 The normal operation is to run the suite and display the
38801
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
88 results. Use -f to save them for later reuse or comparisons.
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
89
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
90 Available timers:
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
91
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
92 time.time
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
93 time.clock
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
94 systimes.processtime
37857
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
95
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
96 Examples:
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
97
38801
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
98 python2.1 pybench.py -f p21.pybench
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
99 python2.5 pybench.py -f p25.pybench
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
100 python pybench.py -s p25.pybench -c p21.pybench
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
101 """
37857
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
102
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
103 License
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
104 -------
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
105
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
106 See LICENSE file.
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
107
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
108
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
109 Sample output
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
110 -------------
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
111
38801
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
112 """
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
113 -------------------------------------------------------------------------------
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
114 PYBENCH 2.0
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
115 -------------------------------------------------------------------------------
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
116 * using Python 2.4.2
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
117 * disabled garbage collection
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
118 * system check interval set to maximum: 2147483647
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
119 * using timer: time.time
37857
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
120
38801
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
121 Calibrating tests. Please wait...
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
122
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
123 Running 10 round(s) of the suite at warp factor 10:
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
124
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
125 * Round 1 done in 6.388 seconds.
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
126 * Round 2 done in 6.485 seconds.
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
127 * Round 3 done in 6.786 seconds.
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
128 ...
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
129 * Round 10 done in 6.546 seconds.
37857
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
130
38801
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
131 -------------------------------------------------------------------------------
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
132 Benchmark: 2006-06-12 12:09:25
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
133 -------------------------------------------------------------------------------
37857
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
134
38801
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
135 Rounds: 10
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
136 Warp: 10
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
137 Timer: time.time
37857
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
138
38801
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
139 Machine Details:
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
140 Platform ID: Linux-2.6.8-24.19-default-x86_64-with-SuSE-9.2-x86-64
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
141 Processor: x86_64
37857
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
142
38801
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
143 Python:
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
144 Executable: /usr/local/bin/python
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
145 Version: 2.4.2
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
146 Compiler: GCC 3.3.4 (pre 3.3.5 20040809)
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
147 Bits: 64bit
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
148 Build: Oct 1 2005 15:24:35 (#1)
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
149 Unicode: UCS2
37857
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
150
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
151
38801
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
152 Test minimum average operation overhead
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
153 -------------------------------------------------------------------------------
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
154 BuiltinFunctionCalls: 126ms 145ms 0.28us 0.274ms
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
155 BuiltinMethodLookup: 124ms 130ms 0.12us 0.316ms
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
156 CompareFloats: 109ms 110ms 0.09us 0.361ms
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
157 CompareFloatsIntegers: 100ms 104ms 0.12us 0.271ms
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
158 CompareIntegers: 137ms 138ms 0.08us 0.542ms
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
159 CompareInternedStrings: 124ms 127ms 0.08us 1.367ms
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
160 CompareLongs: 100ms 104ms 0.10us 0.316ms
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
161 CompareStrings: 111ms 115ms 0.12us 0.929ms
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
162 CompareUnicode: 108ms 128ms 0.17us 0.693ms
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
163 ConcatStrings: 142ms 155ms 0.31us 0.562ms
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
164 ConcatUnicode: 119ms 127ms 0.42us 0.384ms
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
165 CreateInstances: 123ms 128ms 1.14us 0.367ms
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
166 CreateNewInstances: 121ms 126ms 1.49us 0.335ms
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
167 CreateStringsWithConcat: 130ms 135ms 0.14us 0.916ms
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
168 CreateUnicodeWithConcat: 130ms 135ms 0.34us 0.361ms
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
169 DictCreation: 108ms 109ms 0.27us 0.361ms
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
170 DictWithFloatKeys: 149ms 153ms 0.17us 0.678ms
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
171 DictWithIntegerKeys: 124ms 126ms 0.11us 0.915ms
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
172 DictWithStringKeys: 114ms 117ms 0.10us 0.905ms
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
173 ForLoops: 110ms 111ms 4.46us 0.063ms
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
174 IfThenElse: 118ms 119ms 0.09us 0.685ms
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
175 ListSlicing: 116ms 120ms 8.59us 0.103ms
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
176 NestedForLoops: 125ms 137ms 0.09us 0.019ms
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
177 NormalClassAttribute: 124ms 136ms 0.11us 0.457ms
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
178 NormalInstanceAttribute: 110ms 117ms 0.10us 0.454ms
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
179 PythonFunctionCalls: 107ms 113ms 0.34us 0.271ms
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
180 PythonMethodCalls: 140ms 149ms 0.66us 0.141ms
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
181 Recursion: 156ms 166ms 3.32us 0.452ms
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
182 SecondImport: 112ms 118ms 1.18us 0.180ms
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
183 SecondPackageImport: 118ms 127ms 1.27us 0.180ms
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
184 SecondSubmoduleImport: 140ms 151ms 1.51us 0.180ms
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
185 SimpleComplexArithmetic: 128ms 139ms 0.16us 0.361ms
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
186 SimpleDictManipulation: 134ms 136ms 0.11us 0.452ms
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
187 SimpleFloatArithmetic: 110ms 113ms 0.09us 0.571ms
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
188 SimpleIntFloatArithmetic: 106ms 111ms 0.08us 0.548ms
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
189 SimpleIntegerArithmetic: 106ms 109ms 0.08us 0.544ms
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
190 SimpleListManipulation: 103ms 113ms 0.10us 0.587ms
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
191 SimpleLongArithmetic: 112ms 118ms 0.18us 0.271ms
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
192 SmallLists: 105ms 116ms 0.17us 0.366ms
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
193 SmallTuples: 108ms 128ms 0.24us 0.406ms
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
194 SpecialClassAttribute: 119ms 136ms 0.11us 0.453ms
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
195 SpecialInstanceAttribute: 143ms 155ms 0.13us 0.454ms
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
196 StringMappings: 115ms 121ms 0.48us 0.405ms
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
197 StringPredicates: 120ms 129ms 0.18us 2.064ms
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
198 StringSlicing: 111ms 127ms 0.23us 0.781ms
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
199 TryExcept: 125ms 126ms 0.06us 0.681ms
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
200 TryRaiseExcept: 133ms 137ms 2.14us 0.361ms
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
201 TupleSlicing: 117ms 120ms 0.46us 0.066ms
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
202 UnicodeMappings: 156ms 160ms 4.44us 0.429ms
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
203 UnicodePredicates: 117ms 121ms 0.22us 2.487ms
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
204 UnicodeProperties: 115ms 153ms 0.38us 2.070ms
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
205 UnicodeSlicing: 126ms 129ms 0.26us 0.689ms
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
206 -------------------------------------------------------------------------------
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
207 Totals: 6283ms 6673ms
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
208 """
37857
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
209 ________________________________________________________________________
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
210
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
211 Writing New Tests
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
212 ________________________________________________________________________
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
213
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
214 pybench tests are simple modules defining one or more pybench.Test
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
215 subclasses.
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
216
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
217 Writing a test essentially boils down to providing two methods:
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
218 .test() which runs .rounds number of .operations test operations each
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
219 and .calibrate() which does the same except that it doesn't actually
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
220 execute the operations.
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
221
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
222
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
223 Here's an example:
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
224 ------------------
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
225
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
226 from pybench import Test
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
227
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
228 class IntegerCounting(Test):
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
229
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
230 # Version number of the test as float (x.yy); this is important
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
231 # for comparisons of benchmark runs - tests with unequal version
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
232 # number will not get compared.
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
233 version = 1.0
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
234
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
235 # The number of abstract operations done in each round of the
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
236 # test. An operation is the basic unit of what you want to
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
237 # measure. The benchmark will output the amount of run-time per
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
238 # operation. Note that in order to raise the measured timings
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
239 # significantly above noise level, it is often required to repeat
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
240 # sets of operations more than once per test round. The measured
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
241 # overhead per test round should be less than 1 second.
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
242 operations = 20
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
243
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
244 # Number of rounds to execute per test run. This should be
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
245 # adjusted to a figure that results in a test run-time of between
38801
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
246 # 1-2 seconds (at warp 1).
37857
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
247 rounds = 100000
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
248
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
249 def test(self):
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
250
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
251 """ Run the test.
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
252
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
253 The test needs to run self.rounds executing
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
254 self.operations number of operations each.
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
255
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
256 """
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
257 # Init the test
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
258 a = 1
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
259
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
260 # Run test rounds
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
261 #
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
262 # NOTE: Use xrange() for all test loops unless you want to face
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
263 # a 20MB process !
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
264 #
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
265 for i in xrange(self.rounds):
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
266
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
267 # Repeat the operations per round to raise the run-time
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
268 # per operation significantly above the noise level of the
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
269 # for-loop overhead.
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
270
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
271 # Execute 20 operations (a += 1):
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
272 a += 1
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
273 a += 1
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
274 a += 1
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
275 a += 1
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
276 a += 1
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
277 a += 1
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
278 a += 1
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
279 a += 1
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
280 a += 1
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
281 a += 1
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
282 a += 1
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
283 a += 1
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
284 a += 1
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
285 a += 1
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
286 a += 1
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
287 a += 1
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
288 a += 1
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
289 a += 1
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
290 a += 1
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
291 a += 1
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
292
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
293 def calibrate(self):
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
294
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
295 """ Calibrate the test.
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
296
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
297 This method should execute everything that is needed to
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
298 setup and run the test - except for the actual operations
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
299 that you intend to measure. pybench uses this method to
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
300 measure the test implementation overhead.
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
301
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
302 """
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
303 # Init the test
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
304 a = 1
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
305
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
306 # Run test rounds (without actually doing any operation)
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
307 for i in xrange(self.rounds):
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
308
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
309 # Skip the actual execution of the operations, since we
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
310 # only want to measure the test's administration overhead.
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
311 pass
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
312
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
313 Registering a new test module
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
314 -----------------------------
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
315
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
316 To register a test module with pybench, the classes need to be
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
317 imported into the pybench.Setup module. pybench will then scan all the
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
318 symbols defined in that module for subclasses of pybench.Test and
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
319 automatically add them to the benchmark suite.
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
320
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
321
38415
ce638d4b4fc8 Update help text and documentaition.
Steve Holden <steve@holdenweb.com>
parents: 37857
diff changeset
322 Breaking Comparability
ce638d4b4fc8 Update help text and documentaition.
Steve Holden <steve@holdenweb.com>
parents: 37857
diff changeset
323 ----------------------
ce638d4b4fc8 Update help text and documentaition.
Steve Holden <steve@holdenweb.com>
parents: 37857
diff changeset
324
ce638d4b4fc8 Update help text and documentaition.
Steve Holden <steve@holdenweb.com>
parents: 37857
diff changeset
325 If a change is made to any individual test that means it is no
38416
90edaf0d9062 Blasted typos ...
Steve Holden <steve@holdenweb.com>
parents: 38415
diff changeset
326 longer strictly comparable with previous runs, the '.version' class
38415
ce638d4b4fc8 Update help text and documentaition.
Steve Holden <steve@holdenweb.com>
parents: 37857
diff changeset
327 variable should be updated. Therefafter, comparisons with previous
ce638d4b4fc8 Update help text and documentaition.
Steve Holden <steve@holdenweb.com>
parents: 37857
diff changeset
328 versions of the test will list as "n/a" to reflect the change.
ce638d4b4fc8 Update help text and documentaition.
Steve Holden <steve@holdenweb.com>
parents: 37857
diff changeset
329
38801
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
330
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
331 Version History
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
332 ---------------
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
333
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
334 2.0: rewrote parts of pybench which resulted in more repeatable
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
335 timings:
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
336 - made timer a parameter
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
337 - changed the platform default timer to use high-resolution
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
338 timers rather than process timers (which have a much lower
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
339 resolution)
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
340 - added option to select timer
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
341 - added process time timer (using systimes.py)
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
342 - changed to use min() as timing estimator (average
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
343 is still taken as well to provide an idea of the difference)
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
344 - garbage collection is turned off per default
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
345 - sys check interval is set to the highest possible value
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
346 - calibration is now a separate step and done using
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
347 a different strategy that allows measuring the test
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
348 overhead more accurately
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
349 - modified the tests to each give a run-time of between
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
350 100-200ms using warp 10
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
351 - changed default warp factor to 10 (from 20)
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
352 - compared results with timeit.py and confirmed measurements
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
353 - bumped all test versions to 2.0
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
354 - updated platform.py to the latest version
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
355 - changed the output format a bit to make it look
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
356 nicer
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
357 - refactored the APIs somewhat
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
358 1.3+: Steve Holden added the NewInstances test and the filtering
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
359 option during the NeedForSpeed sprint; this also triggered a long
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
360 discussion on how to improve benchmark timing and finally
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
361 resulted in the release of 2.0
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
362 1.3: initial checkin into the Python SVN repository
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
363
2167105123e4 Updated to pybench 2.0.
Marc-André Lemburg <mal@egenix.com>
parents: 38416
diff changeset
364
37857
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
365 Have fun,
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
366 --
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
367 Marc-Andre Lemburg
a4d5cefc2de4 Adding pybench 1.3 to the Tools/ directory.
Marc-André Lemburg <mal@egenix.com>
parents:
diff changeset
368 mal@lemburg.com