cpython-lru_cache-weakref: Lib/pickletools.py annotate

annotate Lib/pickletools.py @ 80661:94de77bd0b4b 2.7

Issue #16194: document imp.load_dynamic problems when read several extension modules from the same binary object file.

author	Andrew Svetlov <andrew.svetlov@gmail.com>
date	Thu, 29 Nov 2012 19:51:28 +0200
parents	cd8d3c2d3bce
children	d234dd21374a

rev	line source
27518 b2b574118ffd minor grammar tweaks Skip Montanaro <skip@pobox.com> parents: 27513 diff changeset	1 '''"Executable documentation" for the pickle module.
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	3 Extensive comments about the pickle protocols and pickle-machine opcodes
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	4 can be found here. Some functions meant for external use:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	5
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	6 genops(pickle)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	7 Generate all the opcodes in a pickle, as (opcode, arg, position) triples.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	8
33004 867fef2b5060 Add argument to docstring Andrew M. Kuchling <amk@amk.ca> parents: 27703 diff changeset	9 dis(pickle, out=None, memo=None, indentlevel=4)
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	10 Print a symbolic disassembly of a pickle.
27518 b2b574118ffd minor grammar tweaks Skip Montanaro <skip@pobox.com> parents: 27513 diff changeset	11 '''
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	12
45273 c7f9f9fdcf82 Complete an open todo on pickletools -- add a pickle optimizer. Raymond Hettinger <python@rcn.com> parents: 34590 diff changeset	13 __all__ = ['dis', 'genops', 'optimize']
33917 25714e0fc89d Essentially SF patch 1061679: add missing __all__ to pickletools.py. Tim Peters <tim.peters@gmail.com> parents: 33004 diff changeset	14
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	15 # Other ideas:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	16 #
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	17 # - A pickle verifier: read a pickle and check it exhaustively for
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	18 # well-formedness. dis() does a lot of this already.
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	19 #
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	20 # - A protocol identifier: examine a pickle and return its protocol number
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	21 # (== the highest .proto attr value among all the opcodes in the pickle).
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	22 # dis() already prints this info at the end.
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	23 #
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	24 # - A pickle optimizer: for example, tuple-building code is sometimes more
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	25 # elaborate than necessary, catering for the possibility that the tuple
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	26 # is recursive. Or lots of times a PUT is generated that's never accessed
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	27 # by a later GET.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	28
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	29
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	30 """
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	31 "A pickle" is a program for a virtual pickle machine (PM, but more accurately
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	32 called an unpickling machine). It's a sequence of opcodes, interpreted by the
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	33 PM, building an arbitrarily complex Python object.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	34
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	35 For the most part, the PM is very simple: there are no looping, testing, or
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	36 conditional instructions, no arithmetic and no function calls. Opcodes are
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	37 executed once each, from first to last, until a STOP opcode is reached.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	38
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	39 The PM has two data areas, "the stack" and "the memo".
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	40
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	41 Many opcodes push Python objects onto the stack; e.g., INT pushes a Python
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	42 integer object on the stack, whose value is gotten from a decimal string
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	43 literal immediately following the INT opcode in the pickle bytestream. Other
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	44 opcodes take Python objects off the stack. The result of unpickling is
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	45 whatever object is left on the stack when the final STOP opcode is executed.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	46
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	47 The memo is simply an array of objects, or it can be implemented as a dict
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	48 mapping little integers to objects. The memo serves as the PM's "long term
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	49 memory", and the little integers indexing the memo are akin to variable
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	50 names. Some opcodes pop a stack object into the memo at a given index,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	51 and others push a memo object at a given index onto the stack again.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	52
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	53 At heart, that's all the PM has. Subtleties arise for these reasons:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	54
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	55 + Object identity. Objects can be arbitrarily complex, and subobjects
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	56 may be shared (for example, the list [a, a] refers to the same object a
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	57 twice). It can be vital that unpickling recreate an isomorphic object
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	58 graph, faithfully reproducing sharing.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	59
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	60 + Recursive objects. For example, after "L = []; L.append(L)", L is a
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	61 list, and L[0] is the same list. This is related to the object identity
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	62 point, and some sequences of pickle opcodes are subtle in order to
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	63 get the right result in all cases.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	64
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	65 + Things pickle doesn't know everything about. Examples of things pickle
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	66 does know everything about are Python's builtin scalar and container
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	67 types, like ints and tuples. They generally have opcodes dedicated to
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	68 them. For things like module references and instances of user-defined
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	69 classes, pickle's knowledge is limited. Historically, many enhancements
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	70 have been made to the pickle protocol in order to do a better (faster,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	71 and/or more compact) job on those.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	72
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	73 + Backward compatibility and micro-optimization. As explained below,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	74 pickle opcodes never go away, not even when better ways to do a thing
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	75 get invented. The repertoire of the PM just keeps growing over time.
27462 26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	76 For example, protocol 0 had two opcodes for building Python integers (INT
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	77 and LONG), protocol 1 added three more for more-efficient pickling of short
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	78 integers, and protocol 2 added two more for more-efficient pickling of
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	79 long integers (before protocol 2, the only ways to pickle a Python long
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	80 took time quadratic in the number of digits, for both pickling and
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	81 unpickling). "Opcode bloat" isn't so much a subtlety as a source of
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	82 wearying complication.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	83
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	84
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	85 Pickle protocols:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	86
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	87 For compatibility, the meaning of a pickle opcode never changes. Instead new
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	88 pickle opcodes get added, and each version's unpickler can handle all the
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	89 pickle opcodes in all protocol versions to date. So old pickles continue to
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	90 be readable forever. The pickler can generally be told to restrict itself to
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	91 the subset of opcodes available under previous protocol versions too, so that
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	92 users can create pickles under the current version readable by older
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	93 versions. However, a pickle does not contain its version number embedded
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	94 within it. If an older unpickler tries to read a pickle using a later
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	95 protocol, the result is most likely an exception due to seeing an unknown (in
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	96 the older unpickler) opcode.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	97
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	98 The original pickle used what's now called "protocol 0", and what was called
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	99 "text mode" before Python 2.3. The entire pickle bytestream is made up of
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	100 printable 7-bit ASCII characters, plus the newline character, in protocol 0.
27462 26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	101 That's why it was called text mode. Protocol 0 is small and elegant, but
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	102 sometimes painfully inefficient.
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	103
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	104 The second major set of additions is now called "protocol 1", and was called
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	105 "binary mode" before Python 2.3. This added many opcodes with arguments
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	106 consisting of arbitrary bytes, including NUL bytes and unprintable "high bit"
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	107 bytes. Binary mode pickles can be substantially smaller than equivalent
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	108 text mode pickles, and sometimes faster too; e.g., BININT represents a 4-byte
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	109 int as 4 bytes following the opcode, which is cheaper to unpickle than the
27462 26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	110 (perhaps) 11-character decimal string attached to INT. Protocol 1 also added
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	111 a number of opcodes that operate on many stack elements at once (like APPENDS
27463 2ee9742aadd8 Comments. Tim Peters <tim.peters@gmail.com> parents: 27462 diff changeset	112 and SETITEMS), and "shortcut" opcodes (like EMPTY_DICT and EMPTY_TUPLE).
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	113
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	114 The third major set of additions came in Python 2.3, and is called "protocol
27462 26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	115 2". This added:
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	116
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	117 - A better way to pickle instances of new-style classes (NEWOBJ).
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	118
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	119 - A way for a pickle to identify its protocol (PROTO).
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	120
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	121 - Time- and space- efficient pickling of long ints (LONG{1,4}).
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	122
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	123 - Shortcuts for small tuples (TUPLE{1,2,3}}.
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	124
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	125 - Dedicated opcodes for bools (NEWTRUE, NEWFALSE).
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	126
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	127 - The "extension registry", a vector of popular objects that can be pushed
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	128 efficiently by index (EXT{1,2,4}). This is akin to the memo and GET, but
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	129 the registry contents are predefined (there's nothing akin to the memo's
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	130 PUT).
27513 b1d1fe7fb514 Document the demise of all pretenses of safety, and the difference Guido van Rossum <guido@python.org> parents: 27507 diff changeset	131
27518 b2b574118ffd minor grammar tweaks Skip Montanaro <skip@pobox.com> parents: 27513 diff changeset	132 Another independent change with Python 2.3 is the abandonment of any
b2b574118ffd minor grammar tweaks Skip Montanaro <skip@pobox.com> parents: 27513 diff changeset	133 pretense that it might be safe to load pickles received from untrusted
27513 b1d1fe7fb514 Document the demise of all pretenses of safety, and the difference Guido van Rossum <guido@python.org> parents: 27507 diff changeset	134 parties -- no sufficient security analysis has been done to guarantee
27518 b2b574118ffd minor grammar tweaks Skip Montanaro <skip@pobox.com> parents: 27513 diff changeset	135 this and there isn't a use case that warrants the expense of such an
27513 b1d1fe7fb514 Document the demise of all pretenses of safety, and the difference Guido van Rossum <guido@python.org> parents: 27507 diff changeset	136 analysis.
b1d1fe7fb514 Document the demise of all pretenses of safety, and the difference Guido van Rossum <guido@python.org> parents: 27507 diff changeset	137
b1d1fe7fb514 Document the demise of all pretenses of safety, and the difference Guido van Rossum <guido@python.org> parents: 27507 diff changeset	138 To this end, all tests for __safe_for_unpickling__ or for
47189 f419b2e96a98 Revert copy_reg -> copyreg rename. Georg Brandl <georg@python.org> parents: 47108 diff changeset	139 copy_reg.safe_constructors are removed from the unpickling code.
27513 b1d1fe7fb514 Document the demise of all pretenses of safety, and the difference Guido van Rossum <guido@python.org> parents: 27507 diff changeset	140 References to these variables in the descriptions below are to be seen
b1d1fe7fb514 Document the demise of all pretenses of safety, and the difference Guido van Rossum <guido@python.org> parents: 27507 diff changeset	141 as describing unpickling in Python 2.2 and before.
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	142 """
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	143
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	144 # Meta-rule: Descriptions are stored in instances of descriptor objects,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	145 # with plain constructors. No meta-language is defined from which
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	146 # descriptors could be constructed. If you want, e.g., XML, write a little
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	147 # program to generate XML from the objects.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	148
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	149 ##############################################################################
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	150 # Some pickle opcodes have an argument, following the opcode in the
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	151 # bytestream. An argument is of a specific type, described by an instance
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	152 # of ArgumentDescriptor. These are not to be confused with arguments taken
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	153 # off the stack -- ArgumentDescriptor applies only to arguments embedded in
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	154 # the opcode stream, immediately following an opcode.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	155
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	156 # Represents the number of bytes consumed by an argument delimited by the
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	157 # next newline character.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	158 UP_TO_NEWLINE = -1
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	159
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	160 # Represents the number of bytes consumed by a two-argument opcode where
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	161 # the first argument gives the number of bytes in the second argument.
27435 430ed1c8e071 Removed the new LONG2 opcode: it's extravagant. If LONG1 isn't enough, Tim Peters <tim.peters@gmail.com> parents: 27434 diff changeset	162 TAKEN_FROM_ARGUMENT1 = -2 # num bytes is 1-byte unsigned int
430ed1c8e071 Removed the new LONG2 opcode: it's extravagant. If LONG1 isn't enough, Tim Peters <tim.peters@gmail.com> parents: 27434 diff changeset	163 TAKEN_FROM_ARGUMENT4 = -3 # num bytes is 4-byte signed little-endian int
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	164
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	165 class ArgumentDescriptor(object):
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	166 __slots__ = (
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	167 # name of descriptor record, also a module global name; a string
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	168 'name',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	169
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	170 # length of argument, in bytes; an int; UP_TO_NEWLINE and
27435 430ed1c8e071 Removed the new LONG2 opcode: it's extravagant. If LONG1 isn't enough, Tim Peters <tim.peters@gmail.com> parents: 27434 diff changeset	171 # TAKEN_FROM_ARGUMENT{1,4} are negative values for variable-length
430ed1c8e071 Removed the new LONG2 opcode: it's extravagant. If LONG1 isn't enough, Tim Peters <tim.peters@gmail.com> parents: 27434 diff changeset	172 # cases
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	173 'n',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	174
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	175 # a function taking a file-like object, reading this kind of argument
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	176 # from the object at the current position, advancing the current
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	177 # position by n bytes, and returning the value of the argument
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	178 'reader',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	179
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	180 # human-readable docs for this arg descriptor; a string
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	181 'doc',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	182 )
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	183
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	184 def __init__(self, name, n, reader, doc):
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	185 assert isinstance(name, str)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	186 self.name = name
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	187
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	188 assert isinstance(n, int) and (n >= 0 or
27435 430ed1c8e071 Removed the new LONG2 opcode: it's extravagant. If LONG1 isn't enough, Tim Peters <tim.peters@gmail.com> parents: 27434 diff changeset	189 n in (UP_TO_NEWLINE,
430ed1c8e071 Removed the new LONG2 opcode: it's extravagant. If LONG1 isn't enough, Tim Peters <tim.peters@gmail.com> parents: 27434 diff changeset	190 TAKEN_FROM_ARGUMENT1,
430ed1c8e071 Removed the new LONG2 opcode: it's extravagant. If LONG1 isn't enough, Tim Peters <tim.peters@gmail.com> parents: 27434 diff changeset	191 TAKEN_FROM_ARGUMENT4))
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	192 self.n = n
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	193
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	194 self.reader = reader
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	195
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	196 assert isinstance(doc, str)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	197 self.doc = doc
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	198
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	199 from struct import unpack as _unpack
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	200
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	201 def read_uint1(f):
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	202 r"""
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	203 >>> import StringIO
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	204 >>> read_uint1(StringIO.StringIO('\xff'))
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	205 255
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	206 """
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	207
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	208 data = f.read(1)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	209 if data:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	210 return ord(data)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	211 raise ValueError("not enough data in stream to read uint1")
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	212
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	213 uint1 = ArgumentDescriptor(
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	214 name='uint1',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	215 n=1,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	216 reader=read_uint1,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	217 doc="One-byte unsigned integer.")
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	218
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	219
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	220 def read_uint2(f):
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	221 r"""
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	222 >>> import StringIO
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	223 >>> read_uint2(StringIO.StringIO('\xff\x00'))
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	224 255
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	225 >>> read_uint2(StringIO.StringIO('\xff\xff'))
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	226 65535
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	227 """
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	228
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	229 data = f.read(2)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	230 if len(data) == 2:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	231 return _unpack("<H", data)[0]
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	232 raise ValueError("not enough data in stream to read uint2")
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	233
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	234 uint2 = ArgumentDescriptor(
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	235 name='uint2',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	236 n=2,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	237 reader=read_uint2,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	238 doc="Two-byte unsigned integer, little-endian.")
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	239
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	240
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	241 def read_int4(f):
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	242 r"""
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	243 >>> import StringIO
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	244 >>> read_int4(StringIO.StringIO('\xff\x00\x00\x00'))
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	245 255
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	246 >>> read_int4(StringIO.StringIO('\x00\x00\x00\x80')) == -(2**31)
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	247 True
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	248 """
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	249
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	250 data = f.read(4)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	251 if len(data) == 4:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	252 return _unpack("<i", data)[0]
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	253 raise ValueError("not enough data in stream to read int4")
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	254
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	255 int4 = ArgumentDescriptor(
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	256 name='int4',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	257 n=4,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	258 reader=read_int4,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	259 doc="Four-byte signed integer, little-endian, 2's complement.")
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	260
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	261
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	262 def read_stringnl(f, decode=True, stripquotes=True):
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	263 r"""
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	264 >>> import StringIO
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	265 >>> read_stringnl(StringIO.StringIO("'abcd'\nefg\n"))
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	266 'abcd'
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	267
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	268 >>> read_stringnl(StringIO.StringIO("\n"))
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	269 Traceback (most recent call last):
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	270 ...
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	271 ValueError: no string quotes around ''
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	272
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	273 >>> read_stringnl(StringIO.StringIO("\n"), stripquotes=False)
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	274 ''
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	275
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	276 >>> read_stringnl(StringIO.StringIO("''\n"))
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	277 ''
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	278
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	279 >>> read_stringnl(StringIO.StringIO('"abcd"'))
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	280 Traceback (most recent call last):
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	281 ...
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	282 ValueError: no newline found when trying to read stringnl
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	283
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	284 Embedded escapes are undone in the result.
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	285 >>> read_stringnl(StringIO.StringIO(r"'a\n\\b\x00c\td'" + "\n'e'"))
154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	286 'a\n\\b\x00c\td'
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	287 """
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	288
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	289 data = f.readline()
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	290 if not data.endswith('\n'):
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	291 raise ValueError("no newline found when trying to read stringnl")
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	292 data = data[:-1] # lose the newline
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	293
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	294 if stripquotes:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	295 for q in "'\"":
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	296 if data.startswith(q):
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	297 if not data.endswith(q):
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	298 raise ValueError("strinq quote %r not found at both "
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	299 "ends of %r" % (q, data))
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	300 data = data[1:-1]
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	301 break
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	302 else:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	303 raise ValueError("no string quotes around %r" % data)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	304
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	305 # I'm not sure when 'string_escape' was added to the std codecs; it's
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	306 # crazy not to use it if it's there.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	307 if decode:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	308 data = data.decode('string_escape')
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	309 return data
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	310
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	311 stringnl = ArgumentDescriptor(
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	312 name='stringnl',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	313 n=UP_TO_NEWLINE,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	314 reader=read_stringnl,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	315 doc="""A newline-terminated string.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	316
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	317 This is a repr-style string, with embedded escapes, and
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	318 bracketing quotes.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	319 """)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	320
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	321 def read_stringnl_noescape(f):
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	322 return read_stringnl(f, decode=False, stripquotes=False)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	323
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	324 stringnl_noescape = ArgumentDescriptor(
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	325 name='stringnl_noescape',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	326 n=UP_TO_NEWLINE,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	327 reader=read_stringnl_noescape,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	328 doc="""A newline-terminated string.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	329
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	330 This is a str-style string, without embedded escapes,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	331 or bracketing quotes. It should consist solely of
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	332 printable ASCII characters.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	333 """)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	334
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	335 def read_stringnl_noescape_pair(f):
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	336 r"""
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	337 >>> import StringIO
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	338 >>> read_stringnl_noescape_pair(StringIO.StringIO("Queue\nEmpty\njunk"))
27422 24c8fbec761f stringnl_noescape_pair(): changed the reader to separate the two Tim Peters <tim.peters@gmail.com> parents: 27421 diff changeset	339 'Queue Empty'
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	340 """
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	341
27422 24c8fbec761f stringnl_noescape_pair(): changed the reader to separate the two Tim Peters <tim.peters@gmail.com> parents: 27421 diff changeset	342 return "%s %s" % (read_stringnl_noescape(f), read_stringnl_noescape(f))
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	343
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	344 stringnl_noescape_pair = ArgumentDescriptor(
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	345 name='stringnl_noescape_pair',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	346 n=UP_TO_NEWLINE,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	347 reader=read_stringnl_noescape_pair,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	348 doc="""A pair of newline-terminated strings.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	349
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	350 These are str-style strings, without embedded
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	351 escapes, or bracketing quotes. They should
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	352 consist solely of printable ASCII characters.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	353 The pair is returned as a single string, with
27422 24c8fbec761f stringnl_noescape_pair(): changed the reader to separate the two Tim Peters <tim.peters@gmail.com> parents: 27421 diff changeset	354 a single blank separating the two strings.
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	355 """)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	356
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	357 def read_string4(f):
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	358 r"""
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	359 >>> import StringIO
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	360 >>> read_string4(StringIO.StringIO("\x00\x00\x00\x00abc"))
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	361 ''
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	362 >>> read_string4(StringIO.StringIO("\x03\x00\x00\x00abcdef"))
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	363 'abc'
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	364 >>> read_string4(StringIO.StringIO("\x00\x00\x00\x03abcdef"))
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	365 Traceback (most recent call last):
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	366 ...
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	367 ValueError: expected 50331648 bytes in a string4, but only 6 remain
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	368 """
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	369
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	370 n = read_int4(f)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	371 if n < 0:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	372 raise ValueError("string4 byte count < 0: %d" % n)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	373 data = f.read(n)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	374 if len(data) == n:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	375 return data
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	376 raise ValueError("expected %d bytes in a string4, but only %d remain" %
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	377 (n, len(data)))
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	378
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	379 string4 = ArgumentDescriptor(
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	380 name="string4",
27435 430ed1c8e071 Removed the new LONG2 opcode: it's extravagant. If LONG1 isn't enough, Tim Peters <tim.peters@gmail.com> parents: 27434 diff changeset	381 n=TAKEN_FROM_ARGUMENT4,
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	382 reader=read_string4,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	383 doc="""A counted string.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	384
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	385 The first argument is a 4-byte little-endian signed int giving
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	386 the number of bytes in the string, and the second argument is
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	387 that many bytes.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	388 """)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	389
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	390
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	391 def read_string1(f):
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	392 r"""
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	393 >>> import StringIO
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	394 >>> read_string1(StringIO.StringIO("\x00"))
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	395 ''
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	396 >>> read_string1(StringIO.StringIO("\x03abcdef"))
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	397 'abc'
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	398 """
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	399
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	400 n = read_uint1(f)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	401 assert n >= 0
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	402 data = f.read(n)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	403 if len(data) == n:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	404 return data
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	405 raise ValueError("expected %d bytes in a string1, but only %d remain" %
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	406 (n, len(data)))
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	407
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	408 string1 = ArgumentDescriptor(
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	409 name="string1",
27435 430ed1c8e071 Removed the new LONG2 opcode: it's extravagant. If LONG1 isn't enough, Tim Peters <tim.peters@gmail.com> parents: 27434 diff changeset	410 n=TAKEN_FROM_ARGUMENT1,
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	411 reader=read_string1,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	412 doc="""A counted string.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	413
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	414 The first argument is a 1-byte unsigned int giving the number
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	415 of bytes in the string, and the second argument is that many
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	416 bytes.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	417 """)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	418
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	419
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	420 def read_unicodestringnl(f):
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	421 r"""
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	422 >>> import StringIO
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	423 >>> read_unicodestringnl(StringIO.StringIO("abc\uabcd\njunk"))
154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	424 u'abc\uabcd'
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	425 """
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	426
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	427 data = f.readline()
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	428 if not data.endswith('\n'):
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	429 raise ValueError("no newline found when trying to read "
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	430 "unicodestringnl")
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	431 data = data[:-1] # lose the newline
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	432 return unicode(data, 'raw-unicode-escape')
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	433
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	434 unicodestringnl = ArgumentDescriptor(
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	435 name='unicodestringnl',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	436 n=UP_TO_NEWLINE,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	437 reader=read_unicodestringnl,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	438 doc="""A newline-terminated Unicode string.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	439
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	440 This is raw-unicode-escape encoded, so consists of
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	441 printable ASCII characters, and may contain embedded
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	442 escape sequences.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	443 """)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	444
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	445 def read_unicodestring4(f):
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	446 r"""
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	447 >>> import StringIO
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	448 >>> s = u'abcd\uabcd'
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	449 >>> enc = s.encode('utf-8')
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	450 >>> enc
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	451 'abcd\xea\xaf\x8d'
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	452 >>> n = chr(len(enc)) + chr(0) * 3 # little-endian 4-byte length
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	453 >>> t = read_unicodestring4(StringIO.StringIO(n + enc + 'junk'))
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	454 >>> s == t
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	455 True
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	456
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	457 >>> read_unicodestring4(StringIO.StringIO(n + enc[:-1]))
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	458 Traceback (most recent call last):
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	459 ...
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	460 ValueError: expected 7 bytes in a unicodestring4, but only 6 remain
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	461 """
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	462
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	463 n = read_int4(f)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	464 if n < 0:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	465 raise ValueError("unicodestring4 byte count < 0: %d" % n)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	466 data = f.read(n)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	467 if len(data) == n:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	468 return unicode(data, 'utf-8')
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	469 raise ValueError("expected %d bytes in a unicodestring4, but only %d "
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	470 "remain" % (n, len(data)))
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	471
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	472 unicodestring4 = ArgumentDescriptor(
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	473 name="unicodestring4",
27435 430ed1c8e071 Removed the new LONG2 opcode: it's extravagant. If LONG1 isn't enough, Tim Peters <tim.peters@gmail.com> parents: 27434 diff changeset	474 n=TAKEN_FROM_ARGUMENT4,
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	475 reader=read_unicodestring4,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	476 doc="""A counted Unicode string.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	477
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	478 The first argument is a 4-byte little-endian signed int
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	479 giving the number of bytes in the string, and the second
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	480 argument-- the UTF-8 encoding of the Unicode string --
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	481 contains that many bytes.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	482 """)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	483
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	484
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	485 def read_decimalnl_short(f):
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	486 r"""
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	487 >>> import StringIO
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	488 >>> read_decimalnl_short(StringIO.StringIO("1234\n56"))
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	489 1234
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	490
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	491 >>> read_decimalnl_short(StringIO.StringIO("1234L\n56"))
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	492 Traceback (most recent call last):
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	493 ...
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	494 ValueError: trailing 'L' not allowed in '1234L'
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	495 """
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	496
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	497 s = read_stringnl(f, decode=False, stripquotes=False)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	498 if s.endswith("L"):
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	499 raise ValueError("trailing 'L' not allowed in %r" % s)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	500
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	501 # It's not necessarily true that the result fits in a Python short int:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	502 # the pickle may have been written on a 64-bit box. There's also a hack
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	503 # for True and False here.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	504 if s == "00":
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	505 return False
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	506 elif s == "01":
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	507 return True
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	508
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	509 try:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	510 return int(s)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	511 except OverflowError:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	512 return long(s)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	513
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	514 def read_decimalnl_long(f):
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	515 r"""
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	516 >>> import StringIO
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	517
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	518 >>> read_decimalnl_long(StringIO.StringIO("1234\n56"))
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	519 Traceback (most recent call last):
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	520 ...
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	521 ValueError: trailing 'L' required in '1234'
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	522
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	523 Someday the trailing 'L' will probably go away from this output.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	524
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	525 >>> read_decimalnl_long(StringIO.StringIO("1234L\n56"))
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	526 1234L
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	527
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	528 >>> read_decimalnl_long(StringIO.StringIO("123456789012345678901234L\n6"))
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	529 123456789012345678901234L
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	530 """
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	531
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	532 s = read_stringnl(f, decode=False, stripquotes=False)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	533 if not s.endswith("L"):
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	534 raise ValueError("trailing 'L' required in %r" % s)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	535 return long(s)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	536
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	537
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	538 decimalnl_short = ArgumentDescriptor(
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	539 name='decimalnl_short',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	540 n=UP_TO_NEWLINE,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	541 reader=read_decimalnl_short,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	542 doc="""A newline-terminated decimal integer literal.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	543
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	544 This never has a trailing 'L', and the integer fit
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	545 in a short Python int on the box where the pickle
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	546 was written -- but there's no guarantee it will fit
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	547 in a short Python int on the box where the pickle
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	548 is read.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	549 """)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	550
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	551 decimalnl_long = ArgumentDescriptor(
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	552 name='decimalnl_long',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	553 n=UP_TO_NEWLINE,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	554 reader=read_decimalnl_long,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	555 doc="""A newline-terminated decimal integer literal.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	556
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	557 This has a trailing 'L', and can represent integers
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	558 of any size.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	559 """)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	560
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	561
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	562 def read_floatnl(f):
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	563 r"""
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	564 >>> import StringIO
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	565 >>> read_floatnl(StringIO.StringIO("-1.25\n6"))
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	566 -1.25
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	567 """
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	568 s = read_stringnl(f, decode=False, stripquotes=False)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	569 return float(s)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	570
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	571 floatnl = ArgumentDescriptor(
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	572 name='floatnl',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	573 n=UP_TO_NEWLINE,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	574 reader=read_floatnl,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	575 doc="""A newline-terminated decimal floating literal.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	576
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	577 In general this requires 17 significant digits for roundtrip
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	578 identity, and pickling then unpickling infinities, NaNs, and
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	579 minus zero doesn't work across boxes, or on some boxes even
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	580 on itself (e.g., Windows can't read the strings it produces
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	581 for infinities or NaNs).
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	582 """)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	583
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	584 def read_float8(f):
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	585 r"""
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	586 >>> import StringIO, struct
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	587 >>> raw = struct.pack(">d", -1.25)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	588 >>> raw
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	589 '\xbf\xf4\x00\x00\x00\x00\x00\x00'
154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	590 >>> read_float8(StringIO.StringIO(raw + "\n"))
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	591 -1.25
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	592 """
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	593
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	594 data = f.read(8)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	595 if len(data) == 8:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	596 return _unpack(">d", data)[0]
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	597 raise ValueError("not enough data in stream to read float8")
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	598
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	599
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	600 float8 = ArgumentDescriptor(
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	601 name='float8',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	602 n=8,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	603 reader=read_float8,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	604 doc="""An 8-byte binary representation of a float, big-endian.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	605
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	606 The format is unique to Python, and shared with the struct
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	607 module (format string '>d') "in theory" (the struct and cPickle
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	608 implementations don't share the code -- they should). It's
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	609 strongly related to the IEEE-754 double format, and, in normal
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	610 cases, is in fact identical to the big-endian 754 double format.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	611 On other boxes the dynamic range is limited to that of a 754
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	612 double, and "add a half and chop" rounding is used to reduce
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	613 the precision to 53 bits. However, even on a 754 box,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	614 infinities, NaNs, and minus zero may not be handled correctly
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	615 (may not survive roundtrip pickling intact).
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	616 """)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	617
27429 b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	618 # Protocol 2 formats
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	619
27506 b480a4f707f6 pickle: Comment repair. Tim Peters <tim.peters@gmail.com> parents: 27473 diff changeset	620 from pickle import decode_long
27429 b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	621
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	622 def read_long1(f):
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	623 r"""
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	624 >>> import StringIO
27552 50c734250e05 It's Official: for LONG1/LONG4, a "byte count" of 0 is taken as a Tim Peters <tim.peters@gmail.com> parents: 27535 diff changeset	625 >>> read_long1(StringIO.StringIO("\x00"))
50c734250e05 It's Official: for LONG1/LONG4, a "byte count" of 0 is taken as a Tim Peters <tim.peters@gmail.com> parents: 27535 diff changeset	626 0L
27429 b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	627 >>> read_long1(StringIO.StringIO("\x02\xff\x00"))
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	628 255L
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	629 >>> read_long1(StringIO.StringIO("\x02\xff\x7f"))
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	630 32767L
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	631 >>> read_long1(StringIO.StringIO("\x02\x00\xff"))
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	632 -256L
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	633 >>> read_long1(StringIO.StringIO("\x02\x00\x80"))
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	634 -32768L
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	635 """
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	636
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	637 n = read_uint1(f)
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	638 data = f.read(n)
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	639 if len(data) != n:
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	640 raise ValueError("not enough data in stream to read long1")
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	641 return decode_long(data)
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	642
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	643 long1 = ArgumentDescriptor(
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	644 name="long1",
27435 430ed1c8e071 Removed the new LONG2 opcode: it's extravagant. If LONG1 isn't enough, Tim Peters <tim.peters@gmail.com> parents: 27434 diff changeset	645 n=TAKEN_FROM_ARGUMENT1,
27429 b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	646 reader=read_long1,
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	647 doc="""A binary long, little-endian, using 1-byte size.
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	648
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	649 This first reads one byte as an unsigned size, then reads that
27434 7116cb5bac50 Documented the 2's-comp business for the new long opcodes. Tim Peters <tim.peters@gmail.com> parents: 27433 diff changeset	650 many bytes and interprets them as a little-endian 2's-complement long.
27552 50c734250e05 It's Official: for LONG1/LONG4, a "byte count" of 0 is taken as a Tim Peters <tim.peters@gmail.com> parents: 27535 diff changeset	651 If the size is 0, that's taken as a shortcut for the long 0L.
27429 b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	652 """)
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	653
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	654 def read_long4(f):
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	655 r"""
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	656 >>> import StringIO
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	657 >>> read_long4(StringIO.StringIO("\x02\x00\x00\x00\xff\x00"))
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	658 255L
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	659 >>> read_long4(StringIO.StringIO("\x02\x00\x00\x00\xff\x7f"))
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	660 32767L
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	661 >>> read_long4(StringIO.StringIO("\x02\x00\x00\x00\x00\xff"))
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	662 -256L
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	663 >>> read_long4(StringIO.StringIO("\x02\x00\x00\x00\x00\x80"))
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	664 -32768L
27552 50c734250e05 It's Official: for LONG1/LONG4, a "byte count" of 0 is taken as a Tim Peters <tim.peters@gmail.com> parents: 27535 diff changeset	665 >>> read_long1(StringIO.StringIO("\x00\x00\x00\x00"))
50c734250e05 It's Official: for LONG1/LONG4, a "byte count" of 0 is taken as a Tim Peters <tim.peters@gmail.com> parents: 27535 diff changeset	666 0L
27429 b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	667 """
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	668
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	669 n = read_int4(f)
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	670 if n < 0:
27436 60ff1ba428c3 Use proper function name in exception messages Neal Norwitz <nnorwitz@gmail.com> parents: 27435 diff changeset	671 raise ValueError("long4 byte count < 0: %d" % n)
27429 b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	672 data = f.read(n)
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	673 if len(data) != n:
27436 60ff1ba428c3 Use proper function name in exception messages Neal Norwitz <nnorwitz@gmail.com> parents: 27435 diff changeset	674 raise ValueError("not enough data in stream to read long4")
27429 b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	675 return decode_long(data)
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	676
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	677 long4 = ArgumentDescriptor(
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	678 name="long4",
27435 430ed1c8e071 Removed the new LONG2 opcode: it's extravagant. If LONG1 isn't enough, Tim Peters <tim.peters@gmail.com> parents: 27434 diff changeset	679 n=TAKEN_FROM_ARGUMENT4,
27429 b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	680 reader=read_long4,
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	681 doc="""A binary representation of a long, little-endian.
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	682
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	683 This first reads four bytes as a signed size (but requires the
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	684 size to be >= 0), then reads that many bytes and interprets them
27552 50c734250e05 It's Official: for LONG1/LONG4, a "byte count" of 0 is taken as a Tim Peters <tim.peters@gmail.com> parents: 27535 diff changeset	685 as a little-endian 2's-complement long. If the size is 0, that's taken
50c734250e05 It's Official: for LONG1/LONG4, a "byte count" of 0 is taken as a Tim Peters <tim.peters@gmail.com> parents: 27535 diff changeset	686 as a shortcut for the long 0L, although LONG1 should really be used
50c734250e05 It's Official: for LONG1/LONG4, a "byte count" of 0 is taken as a Tim Peters <tim.peters@gmail.com> parents: 27535 diff changeset	687 then instead (and in any case where # of bytes < 256).
27429 b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	688 """)
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	689
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	690
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	691 ##############################################################################
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	692 # Object descriptors. The stack used by the pickle machine holds objects,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	693 # and in the stack_before and stack_after attributes of OpcodeInfo
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	694 # descriptors we need names to describe the various types of objects that can
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	695 # appear on the stack.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	696
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	697 class StackObject(object):
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	698 __slots__ = (
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	699 # name of descriptor record, for info only
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	700 'name',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	701
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	702 # type of object, or tuple of type objects (meaning the object can
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	703 # be of any type in the tuple)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	704 'obtype',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	705
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	706 # human-readable docs for this kind of stack object; a string
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	707 'doc',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	708 )
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	709
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	710 def __init__(self, name, obtype, doc):
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	711 assert isinstance(name, str)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	712 self.name = name
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	713
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	714 assert isinstance(obtype, type) or isinstance(obtype, tuple)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	715 if isinstance(obtype, tuple):
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	716 for contained in obtype:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	717 assert isinstance(contained, type)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	718 self.obtype = obtype
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	719
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	720 assert isinstance(doc, str)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	721 self.doc = doc
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	722
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	723 def __repr__(self):
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	724 return self.name
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	725
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	726
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	727 pyint = StackObject(
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	728 name='int',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	729 obtype=int,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	730 doc="A short (as opposed to long) Python integer object.")
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	731
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	732 pylong = StackObject(
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	733 name='long',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	734 obtype=long,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	735 doc="A long (as opposed to short) Python integer object.")
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	736
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	737 pyinteger_or_bool = StackObject(
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	738 name='int_or_bool',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	739 obtype=(int, long, bool),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	740 doc="A Python integer object (short or long), or "
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	741 "a Python bool.")
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	742
27429 b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	743 pybool = StackObject(
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	744 name='bool',
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	745 obtype=(bool,),
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	746 doc="A Python bool object.")
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	747
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	748 pyfloat = StackObject(
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	749 name='float',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	750 obtype=float,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	751 doc="A Python float object.")
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	752
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	753 pystring = StackObject(
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	754 name='str',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	755 obtype=str,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	756 doc="A Python string object.")
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	757
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	758 pyunicode = StackObject(
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	759 name='unicode',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	760 obtype=unicode,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	761 doc="A Python Unicode string object.")
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	762
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	763 pynone = StackObject(
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	764 name="None",
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	765 obtype=type(None),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	766 doc="The Python None object.")
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	767
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	768 pytuple = StackObject(
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	769 name="tuple",
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	770 obtype=tuple,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	771 doc="A Python tuple object.")
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	772
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	773 pylist = StackObject(
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	774 name="list",
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	775 obtype=list,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	776 doc="A Python list object.")
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	777
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	778 pydict = StackObject(
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	779 name="dict",
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	780 obtype=dict,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	781 doc="A Python dict object.")
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	782
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	783 anyobject = StackObject(
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	784 name='any',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	785 obtype=object,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	786 doc="Any kind of object whatsoever.")
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	787
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	788 markobject = StackObject(
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	789 name="mark",
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	790 obtype=StackObject,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	791 doc="""'The mark' is a unique object.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	792
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	793 Opcodes that operate on a variable number of objects
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	794 generally don't embed the count of objects in the opcode,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	795 or pull it off the stack. Instead the MARK opcode is used
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	796 to push a special marker object on the stack, and then
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	797 some other opcodes grab all the objects from the top of
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	798 the stack down to (but not including) the topmost marker
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	799 object.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	800 """)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	801
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	802 stackslice = StackObject(
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	803 name="stackslice",
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	804 obtype=StackObject,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	805 doc="""An object representing a contiguous slice of the stack.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	806
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	807 This is used in conjuction with markobject, to represent all
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	808 of the stack following the topmost markobject. For example,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	809 the POP_MARK opcode changes the stack from
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	810
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	811 [..., markobject, stackslice]
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	812 to
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	813 [...]
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	814
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	815 No matter how many object are on the stack after the topmost
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	816 markobject, POP_MARK gets rid of all of them (including the
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	817 topmost markobject too).
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	818 """)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	819
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	820 ##############################################################################
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	821 # Descriptors for pickle opcodes.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	822
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	823 class OpcodeInfo(object):
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	824
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	825 __slots__ = (
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	826 # symbolic name of opcode; a string
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	827 'name',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	828
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	829 # the code used in a bytestream to represent the opcode; a
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	830 # one-character string
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	831 'code',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	832
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	833 # If the opcode has an argument embedded in the byte string, an
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	834 # instance of ArgumentDescriptor specifying its type. Note that
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	835 # arg.reader(s) can be used to read and decode the argument from
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	836 # the bytestream s, and arg.doc documents the format of the raw
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	837 # argument bytes. If the opcode doesn't have an argument embedded
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	838 # in the bytestream, arg should be None.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	839 'arg',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	840
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	841 # what the stack looks like before this opcode runs; a list
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	842 'stack_before',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	843
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	844 # what the stack looks like after this opcode runs; a list
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	845 'stack_after',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	846
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	847 # the protocol number in which this opcode was introduced; an int
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	848 'proto',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	849
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	850 # human-readable docs for this opcode; a string
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	851 'doc',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	852 )
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	853
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	854 def __init__(self, name, code, arg,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	855 stack_before, stack_after, proto, doc):
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	856 assert isinstance(name, str)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	857 self.name = name
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	858
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	859 assert isinstance(code, str)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	860 assert len(code) == 1
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	861 self.code = code
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	862
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	863 assert arg is None or isinstance(arg, ArgumentDescriptor)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	864 self.arg = arg
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	865
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	866 assert isinstance(stack_before, list)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	867 for x in stack_before:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	868 assert isinstance(x, StackObject)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	869 self.stack_before = stack_before
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	870
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	871 assert isinstance(stack_after, list)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	872 for x in stack_after:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	873 assert isinstance(x, StackObject)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	874 self.stack_after = stack_after
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	875
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	876 assert isinstance(proto, int) and 0 <= proto <= 2
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	877 self.proto = proto
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	878
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	879 assert isinstance(doc, str)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	880 self.doc = doc
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	881
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	882 I = OpcodeInfo
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	883 opcodes = [
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	884
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	885 # Ways to spell integers.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	886
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	887 I(name='INT',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	888 code='I',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	889 arg=decimalnl_short,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	890 stack_before=[],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	891 stack_after=[pyinteger_or_bool],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	892 proto=0,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	893 doc="""Push an integer or bool.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	894
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	895 The argument is a newline-terminated decimal literal string.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	896
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	897 The intent may have been that this always fit in a short Python int,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	898 but INT can be generated in pickles written on a 64-bit box that
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	899 require a Python long on a 32-bit box. The difference between this
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	900 and LONG then is that INT skips a trailing 'L', and produces a short
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	901 int whenever possible.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	902
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	903 Another difference is due to that, when bool was introduced as a
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	904 distinct type in 2.3, builtin names True and False were also added to
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	905 2.2.2, mapping to ints 1 and 0. For compatibility in both directions,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	906 True gets pickled as INT + "I01\\n", and False as INT + "I00\\n".
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	907 Leading zeroes are never produced for a genuine integer. The 2.3
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	908 (and later) unpicklers special-case these and return bool instead;
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	909 earlier unpicklers ignore the leading "0" and return the int.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	910 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	911
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	912 I(name='BININT',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	913 code='J',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	914 arg=int4,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	915 stack_before=[],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	916 stack_after=[pyint],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	917 proto=1,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	918 doc="""Push a four-byte signed integer.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	919
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	920 This handles the full range of Python (short) integers on a 32-bit
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	921 box, directly as binary bytes (1 for the opcode and 4 for the integer).
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	922 If the integer is non-negative and fits in 1 or 2 bytes, pickling via
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	923 BININT1 or BININT2 saves space.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	924 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	925
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	926 I(name='BININT1',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	927 code='K',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	928 arg=uint1,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	929 stack_before=[],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	930 stack_after=[pyint],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	931 proto=1,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	932 doc="""Push a one-byte unsigned integer.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	933
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	934 This is a space optimization for pickling very small non-negative ints,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	935 in range(256).
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	936 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	937
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	938 I(name='BININT2',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	939 code='M',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	940 arg=uint2,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	941 stack_before=[],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	942 stack_after=[pyint],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	943 proto=1,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	944 doc="""Push a two-byte unsigned integer.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	945
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	946 This is a space optimization for pickling small positive ints, in
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	947 range(256, 2**16). Integers in range(256) can also be pickled via
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	948 BININT2, but BININT1 instead saves a byte.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	949 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	950
27462 26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	951 I(name='LONG',
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	952 code='L',
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	953 arg=decimalnl_long,
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	954 stack_before=[],
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	955 stack_after=[pylong],
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	956 proto=0,
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	957 doc="""Push a long integer.
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	958
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	959 The same as INT, except that the literal ends with 'L', and always
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	960 unpickles to a Python long. There doesn't seem a real purpose to the
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	961 trailing 'L'.
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	962
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	963 Note that LONG takes time quadratic in the number of digits when
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	964 unpickling (this is simply due to the nature of decimal->binary
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	965 conversion). Proto 2 added linear-time (in C; still quadratic-time
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	966 in Python) LONG1 and LONG4 opcodes.
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	967 """),
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	968
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	969 I(name="LONG1",
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	970 code='\x8a',
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	971 arg=long1,
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	972 stack_before=[],
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	973 stack_after=[pylong],
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	974 proto=2,
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	975 doc="""Long integer using one-byte length.
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	976
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	977 A more efficient encoding of a Python long; the long1 encoding
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	978 says it all."""),
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	979
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	980 I(name="LONG4",
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	981 code='\x8b',
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	982 arg=long4,
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	983 stack_before=[],
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	984 stack_after=[pylong],
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	985 proto=2,
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	986 doc="""Long integer using found-byte length.
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	987
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	988 A more efficient encoding of a Python long; the long4 encoding
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	989 says it all."""),
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	990
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	991 # Ways to spell strings (8-bit, not Unicode).
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	992
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	993 I(name='STRING',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	994 code='S',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	995 arg=stringnl,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	996 stack_before=[],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	997 stack_after=[pystring],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	998 proto=0,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	999 doc="""Push a Python string object.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1000
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1001 The argument is a repr-style string, with bracketing quote characters,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1002 and perhaps embedded escapes. The argument extends until the next
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1003 newline character.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1004 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1005
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1006 I(name='BINSTRING',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1007 code='T',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1008 arg=string4,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1009 stack_before=[],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1010 stack_after=[pystring],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1011 proto=1,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1012 doc="""Push a Python string object.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1013
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1014 There are two arguments: the first is a 4-byte little-endian signed int
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1015 giving the number of bytes in the string, and the second is that many
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1016 bytes, which are taken literally as the string content.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1017 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1018
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1019 I(name='SHORT_BINSTRING',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1020 code='U',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1021 arg=string1,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1022 stack_before=[],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1023 stack_after=[pystring],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1024 proto=1,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1025 doc="""Push a Python string object.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1026
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1027 There are two arguments: the first is a 1-byte unsigned int giving
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1028 the number of bytes in the string, and the second is that many bytes,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1029 which are taken literally as the string content.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1030 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1031
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1032 # Ways to spell None.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1033
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1034 I(name='NONE',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1035 code='N',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1036 arg=None,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1037 stack_before=[],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1038 stack_after=[pynone],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1039 proto=0,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1040 doc="Push None on the stack."),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1041
27462 26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1042 # Ways to spell bools, starting with proto 2. See INT for how this was
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1043 # done before proto 2.
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1044
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1045 I(name='NEWTRUE',
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1046 code='\x88',
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1047 arg=None,
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1048 stack_before=[],
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1049 stack_after=[pybool],
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1050 proto=2,
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1051 doc="""True.
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1052
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1053 Push True onto the stack."""),
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1054
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1055 I(name='NEWFALSE',
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1056 code='\x89',
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1057 arg=None,
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1058 stack_before=[],
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1059 stack_after=[pybool],
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1060 proto=2,
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1061 doc="""True.
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1062
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1063 Push False onto the stack."""),
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1064
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1065 # Ways to spell Unicode strings.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1066
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1067 I(name='UNICODE',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1068 code='V',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1069 arg=unicodestringnl,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1070 stack_before=[],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1071 stack_after=[pyunicode],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1072 proto=0, # this may be pure-text, but it's a later addition
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1073 doc="""Push a Python Unicode string object.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1074
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1075 The argument is a raw-unicode-escape encoding of a Unicode string,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1076 and so may contain embedded escape sequences. The argument extends
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1077 until the next newline character.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1078 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1079
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1080 I(name='BINUNICODE',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1081 code='X',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1082 arg=unicodestring4,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1083 stack_before=[],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1084 stack_after=[pyunicode],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1085 proto=1,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1086 doc="""Push a Python Unicode string object.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1087
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1088 There are two arguments: the first is a 4-byte little-endian signed int
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1089 giving the number of bytes in the string. The second is that many
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1090 bytes, and is the UTF-8 encoding of the Unicode string.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1091 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1092
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1093 # Ways to spell floats.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1094
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1095 I(name='FLOAT',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1096 code='F',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1097 arg=floatnl,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1098 stack_before=[],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1099 stack_after=[pyfloat],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1100 proto=0,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1101 doc="""Newline-terminated decimal float literal.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1102
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1103 The argument is repr(a_float), and in general requires 17 significant
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1104 digits for roundtrip conversion to be an identity (this is so for
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1105 IEEE-754 double precision values, which is what Python float maps to
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1106 on most boxes).
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1107
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1108 In general, FLOAT cannot be used to transport infinities, NaNs, or
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1109 minus zero across boxes (or even on a single box, if the platform C
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1110 library can't read the strings it produces for such things -- Windows
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1111 is like that), but may do less damage than BINFLOAT on boxes with
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1112 greater precision or dynamic range than IEEE-754 double.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1113 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1114
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1115 I(name='BINFLOAT',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1116 code='G',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1117 arg=float8,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1118 stack_before=[],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1119 stack_after=[pyfloat],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1120 proto=1,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1121 doc="""Float stored in binary form, with 8 bytes of data.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1122
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1123 This generally requires less than half the space of FLOAT encoding.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1124 In general, BINFLOAT cannot be used to transport infinities, NaNs, or
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1125 minus zero, raises an exception if the exponent exceeds the range of
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1126 an IEEE-754 double, and retains no more than 53 bits of precision (if
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1127 there are more than that, "add a half and chop" rounding is used to
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1128 cut it back to 53 significant bits).
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1129 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1130
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1131 # Ways to build lists.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1132
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1133 I(name='EMPTY_LIST',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1134 code=']',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1135 arg=None,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1136 stack_before=[],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1137 stack_after=[pylist],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1138 proto=1,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1139 doc="Push an empty list."),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1140
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1141 I(name='APPEND',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1142 code='a',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1143 arg=None,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1144 stack_before=[pylist, anyobject],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1145 stack_after=[pylist],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1146 proto=0,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1147 doc="""Append an object to a list.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1148
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1149 Stack before: ... pylist anyobject
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1150 Stack after: ... pylist+[anyobject]
27463 2ee9742aadd8 Comments. Tim Peters <tim.peters@gmail.com> parents: 27462 diff changeset	1151
2ee9742aadd8 Comments. Tim Peters <tim.peters@gmail.com> parents: 27462 diff changeset	1152 although pylist is really extended in-place.
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1153 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1154
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1155 I(name='APPENDS',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1156 code='e',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1157 arg=None,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1158 stack_before=[pylist, markobject, stackslice],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1159 stack_after=[pylist],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1160 proto=1,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1161 doc="""Extend a list by a slice of stack objects.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1162
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1163 Stack before: ... pylist markobject stackslice
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1164 Stack after: ... pylist+stackslice
27463 2ee9742aadd8 Comments. Tim Peters <tim.peters@gmail.com> parents: 27462 diff changeset	1165
2ee9742aadd8 Comments. Tim Peters <tim.peters@gmail.com> parents: 27462 diff changeset	1166 although pylist is really extended in-place.
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1167 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1168
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1169 I(name='LIST',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1170 code='l',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1171 arg=None,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1172 stack_before=[markobject, stackslice],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1173 stack_after=[pylist],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1174 proto=0,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1175 doc="""Build a list out of the topmost stack slice, after markobject.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1176
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1177 All the stack entries following the topmost markobject are placed into
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1178 a single Python list, which single list object replaces all of the
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1179 stack from the topmost markobject onward. For example,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1180
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1181 Stack before: ... markobject 1 2 3 'abc'
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1182 Stack after: ... [1, 2, 3, 'abc']
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1183 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1184
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1185 # Ways to build tuples.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1186
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1187 I(name='EMPTY_TUPLE',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1188 code=')',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1189 arg=None,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1190 stack_before=[],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1191 stack_after=[pytuple],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1192 proto=1,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1193 doc="Push an empty tuple."),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1194
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1195 I(name='TUPLE',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1196 code='t',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1197 arg=None,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1198 stack_before=[markobject, stackslice],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1199 stack_after=[pytuple],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1200 proto=0,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1201 doc="""Build a tuple out of the topmost stack slice, after markobject.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1202
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1203 All the stack entries following the topmost markobject are placed into
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1204 a single Python tuple, which single tuple object replaces all of the
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1205 stack from the topmost markobject onward. For example,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1206
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1207 Stack before: ... markobject 1 2 3 'abc'
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1208 Stack after: ... (1, 2, 3, 'abc')
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1209 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1210
27462 26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1211 I(name='TUPLE1',
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1212 code='\x85',
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1213 arg=None,
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1214 stack_before=[anyobject],
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1215 stack_after=[pytuple],
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1216 proto=2,
63042 e7031983e6fd Merged revisions 82915 via svnmerge from Alexander Belopolsky <alexander.belopolsky@gmail.com> parents: 54230 diff changeset	1217 doc="""Build a one-tuple out of the topmost item on the stack.
27462 26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1218
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1219 This code pops one value off the stack and pushes a tuple of
63042 e7031983e6fd Merged revisions 82915 via svnmerge from Alexander Belopolsky <alexander.belopolsky@gmail.com> parents: 54230 diff changeset	1220 length 1 whose one item is that value back onto it. In other
e7031983e6fd Merged revisions 82915 via svnmerge from Alexander Belopolsky <alexander.belopolsky@gmail.com> parents: 54230 diff changeset	1221 words:
27462 26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1222
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1223 stack[-1] = tuple(stack[-1:])
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1224 """),
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1225
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1226 I(name='TUPLE2',
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1227 code='\x86',
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1228 arg=None,
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1229 stack_before=[anyobject, anyobject],
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1230 stack_after=[pytuple],
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1231 proto=2,
63042 e7031983e6fd Merged revisions 82915 via svnmerge from Alexander Belopolsky <alexander.belopolsky@gmail.com> parents: 54230 diff changeset	1232 doc="""Build a two-tuple out of the top two items on the stack.
27462 26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1233
63042 e7031983e6fd Merged revisions 82915 via svnmerge from Alexander Belopolsky <alexander.belopolsky@gmail.com> parents: 54230 diff changeset	1234 This code pops two values off the stack and pushes a tuple of
e7031983e6fd Merged revisions 82915 via svnmerge from Alexander Belopolsky <alexander.belopolsky@gmail.com> parents: 54230 diff changeset	1235 length 2 whose items are those values back onto it. In other
e7031983e6fd Merged revisions 82915 via svnmerge from Alexander Belopolsky <alexander.belopolsky@gmail.com> parents: 54230 diff changeset	1236 words:
27462 26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1237
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1238 stack[-2:] = [tuple(stack[-2:])]
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1239 """),
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1240
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1241 I(name='TUPLE3',
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1242 code='\x87',
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1243 arg=None,
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1244 stack_before=[anyobject, anyobject, anyobject],
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1245 stack_after=[pytuple],
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1246 proto=2,
63042 e7031983e6fd Merged revisions 82915 via svnmerge from Alexander Belopolsky <alexander.belopolsky@gmail.com> parents: 54230 diff changeset	1247 doc="""Build a three-tuple out of the top three items on the stack.
27462 26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1248
63042 e7031983e6fd Merged revisions 82915 via svnmerge from Alexander Belopolsky <alexander.belopolsky@gmail.com> parents: 54230 diff changeset	1249 This code pops three values off the stack and pushes a tuple of
e7031983e6fd Merged revisions 82915 via svnmerge from Alexander Belopolsky <alexander.belopolsky@gmail.com> parents: 54230 diff changeset	1250 length 3 whose items are those values back onto it. In other
e7031983e6fd Merged revisions 82915 via svnmerge from Alexander Belopolsky <alexander.belopolsky@gmail.com> parents: 54230 diff changeset	1251 words:
27462 26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1252
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1253 stack[-3:] = [tuple(stack[-3:])]
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1254 """),
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1255
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1256 # Ways to build dicts.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1257
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1258 I(name='EMPTY_DICT',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1259 code='}',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1260 arg=None,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1261 stack_before=[],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1262 stack_after=[pydict],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1263 proto=1,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1264 doc="Push an empty dict."),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1265
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1266 I(name='DICT',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1267 code='d',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1268 arg=None,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1269 stack_before=[markobject, stackslice],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1270 stack_after=[pydict],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1271 proto=0,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1272 doc="""Build a dict out of the topmost stack slice, after markobject.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1273
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1274 All the stack entries following the topmost markobject are placed into
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1275 a single Python dict, which single dict object replaces all of the
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1276 stack from the topmost markobject onward. The stack slice alternates
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1277 key, value, key, value, .... For example,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1278
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1279 Stack before: ... markobject 1 2 3 'abc'
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1280 Stack after: ... {1: 2, 3: 'abc'}
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1281 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1282
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1283 I(name='SETITEM',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1284 code='s',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1285 arg=None,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1286 stack_before=[pydict, anyobject, anyobject],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1287 stack_after=[pydict],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1288 proto=0,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1289 doc="""Add a key+value pair to an existing dict.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1290
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1291 Stack before: ... pydict key value
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1292 Stack after: ... pydict
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1293
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1294 where pydict has been modified via pydict[key] = value.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1295 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1296
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1297 I(name='SETITEMS',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1298 code='u',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1299 arg=None,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1300 stack_before=[pydict, markobject, stackslice],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1301 stack_after=[pydict],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1302 proto=1,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1303 doc="""Add an arbitrary number of key+value pairs to an existing dict.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1304
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1305 The slice of the stack following the topmost markobject is taken as
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1306 an alternating sequence of keys and values, added to the dict
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1307 immediately under the topmost markobject. Everything at and after the
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1308 topmost markobject is popped, leaving the mutated dict at the top
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1309 of the stack.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1310
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1311 Stack before: ... pydict markobject key_1 value_1 ... key_n value_n
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1312 Stack after: ... pydict
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1313
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1314 where pydict has been modified via pydict[key_i] = value_i for i in
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1315 1, 2, ..., n, and in that order.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1316 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1317
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1318 # Stack manipulation.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1319
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1320 I(name='POP',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1321 code='0',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1322 arg=None,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1323 stack_before=[anyobject],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1324 stack_after=[],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1325 proto=0,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1326 doc="Discard the top stack item, shrinking the stack by one item."),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1327
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1328 I(name='DUP',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1329 code='2',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1330 arg=None,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1331 stack_before=[anyobject],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1332 stack_after=[anyobject, anyobject],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1333 proto=0,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1334 doc="Push the top stack item onto the stack again, duplicating it."),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1335
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1336 I(name='MARK',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1337 code='(',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1338 arg=None,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1339 stack_before=[],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1340 stack_after=[markobject],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1341 proto=0,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1342 doc="""Push markobject onto the stack.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1343
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1344 markobject is a unique object, used by other opcodes to identify a
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1345 region of the stack containing a variable number of objects for them
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1346 to work on. See markobject.doc for more detail.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1347 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1348
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1349 I(name='POP_MARK',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1350 code='1',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1351 arg=None,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1352 stack_before=[markobject, stackslice],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1353 stack_after=[],
54230 b3326fe632e5 Merged revisions 72792 via svnmerge from Collin Winter <collinw@gmail.com> parents: 51020 diff changeset	1354 proto=1,
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1355 doc="""Pop all the stack objects at and above the topmost markobject.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1356
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1357 When an opcode using a variable number of stack objects is done,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1358 POP_MARK is used to remove those objects, and to remove the markobject
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1359 that delimited their starting position on the stack.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1360 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1361
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1362 # Memo manipulation. There are really only two operations (get and put),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1363 # each in all-text, "short binary", and "long binary" flavors.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1364
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1365 I(name='GET',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1366 code='g',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1367 arg=decimalnl_short,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1368 stack_before=[],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1369 stack_after=[anyobject],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1370 proto=0,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1371 doc="""Read an object from the memo and push it on the stack.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1372
68588 cd8d3c2d3bce #11565: Fix several typos. Patch by Piotr Kasprzyk. Ezio Melotti <ezio.melotti@gmail.com> parents: 63042 diff changeset	1373 The index of the memo object to push is given by the newline-terminated
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1374 decimal string following. BINGET and LONG_BINGET are space-optimized
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1375 versions.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1376 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1377
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1378 I(name='BINGET',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1379 code='h',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1380 arg=uint1,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1381 stack_before=[],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1382 stack_after=[anyobject],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1383 proto=1,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1384 doc="""Read an object from the memo and push it on the stack.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1385
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1386 The index of the memo object to push is given by the 1-byte unsigned
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1387 integer following.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1388 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1389
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1390 I(name='LONG_BINGET',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1391 code='j',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1392 arg=int4,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1393 stack_before=[],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1394 stack_after=[anyobject],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1395 proto=1,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1396 doc="""Read an object from the memo and push it on the stack.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1397
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1398 The index of the memo object to push is given by the 4-byte signed
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1399 little-endian integer following.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1400 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1401
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1402 I(name='PUT',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1403 code='p',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1404 arg=decimalnl_short,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1405 stack_before=[],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1406 stack_after=[],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1407 proto=0,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1408 doc="""Store the stack top into the memo. The stack is not popped.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1409
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1410 The index of the memo location to write into is given by the newline-
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1411 terminated decimal string following. BINPUT and LONG_BINPUT are
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1412 space-optimized versions.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1413 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1414
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1415 I(name='BINPUT',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1416 code='q',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1417 arg=uint1,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1418 stack_before=[],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1419 stack_after=[],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1420 proto=1,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1421 doc="""Store the stack top into the memo. The stack is not popped.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1422
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1423 The index of the memo location to write into is given by the 1-byte
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1424 unsigned integer following.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1425 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1426
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1427 I(name='LONG_BINPUT',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1428 code='r',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1429 arg=int4,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1430 stack_before=[],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1431 stack_after=[],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1432 proto=1,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1433 doc="""Store the stack top into the memo. The stack is not popped.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1434
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1435 The index of the memo location to write into is given by the 4-byte
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1436 signed little-endian integer following.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1437 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1438
27462 26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1439 # Access the extension registry (predefined objects). Akin to the GET
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1440 # family.
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1441
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1442 I(name='EXT1',
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1443 code='\x82',
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1444 arg=uint1,
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1445 stack_before=[],
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1446 stack_after=[anyobject],
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1447 proto=2,
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1448 doc="""Extension code.
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1449
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1450 This code and the similar EXT2 and EXT4 allow using a registry
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1451 of popular objects that are pickled by name, typically classes.
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1452 It is envisioned that through a global negotiation and
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1453 registration process, third parties can set up a mapping between
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1454 ints and object names.
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1455
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1456 In order to guarantee pickle interchangeability, the extension
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1457 code registry ought to be global, although a range of codes may
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1458 be reserved for private use.
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1459
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1460 EXT1 has a 1-byte integer argument. This is used to index into the
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1461 extension registry, and the object at that index is pushed on the stack.
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1462 """),
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1463
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1464 I(name='EXT2',
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1465 code='\x83',
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1466 arg=uint2,
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1467 stack_before=[],
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1468 stack_after=[anyobject],
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1469 proto=2,
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1470 doc="""Extension code.
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1471
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1472 See EXT1. EXT2 has a two-byte integer argument.
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1473 """),
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1474
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1475 I(name='EXT4',
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1476 code='\x84',
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1477 arg=int4,
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1478 stack_before=[],
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1479 stack_after=[anyobject],
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1480 proto=2,
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1481 doc="""Extension code.
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1482
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1483 See EXT1. EXT4 has a four-byte integer argument.
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1484 """),
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1485
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1486 # Push a class object, or module function, on the stack, via its module
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1487 # and name.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1488
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1489 I(name='GLOBAL',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1490 code='c',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1491 arg=stringnl_noescape_pair,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1492 stack_before=[],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1493 stack_after=[anyobject],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1494 proto=0,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1495 doc="""Push a global object (module.attr) on the stack.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1496
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1497 Two newline-terminated strings follow the GLOBAL opcode. The first is
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1498 taken as a module name, and the second as a class name. The class
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1499 object module.class is pushed on the stack. More accurately, the
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1500 object returned by self.find_class(module, class) is pushed on the
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1501 stack, so unpickling subclasses can override this form of lookup.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1502 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1503
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1504 # Ways to build objects of classes pickle doesn't know about directly
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1505 # (user-defined classes). I despair of documenting this accurately
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1506 # and comprehensibly -- you really have to read the pickle code to
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1507 # find all the special cases.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1508
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1509 I(name='REDUCE',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1510 code='R',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1511 arg=None,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1512 stack_before=[anyobject, anyobject],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1513 stack_after=[anyobject],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1514 proto=0,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1515 doc="""Push an object built from a callable and an argument tuple.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1516
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1517 The opcode is named to remind of the __reduce__() method.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1518
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1519 Stack before: ... callable pytuple
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1520 Stack after: ... callable(*pytuple)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1521
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1522 The callable and the argument tuple are the first two items returned
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1523 by a __reduce__ method. Applying the callable to the argtuple is
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1524 supposed to reproduce the original object, or at least get it started.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1525 If the __reduce__ method returns a 3-tuple, the last component is an
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1526 argument to be passed to the object's __setstate__, and then the REDUCE
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1527 opcode is followed by code to create setstate's argument, and then a
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1528 BUILD opcode to apply __setstate__ to that argument.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1529
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1530 If type(callable) is not ClassType, REDUCE complains unless the
47189 f419b2e96a98 Revert copy_reg -> copyreg rename. Georg Brandl <georg@python.org> parents: 47108 diff changeset	1531 callable has been registered with the copy_reg module's
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1532 safe_constructors dict, or the callable has a magic
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1533 '__safe_for_unpickling__' attribute with a true value. I'm not sure
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1534 why it does this, but I've sure seen this complaint often enough when
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1535 I didn't want to <wink>.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1536 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1537
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1538 I(name='BUILD',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1539 code='b',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1540 arg=None,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1541 stack_before=[anyobject, anyobject],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1542 stack_after=[anyobject],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1543 proto=0,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1544 doc="""Finish building an object, via __setstate__ or dict update.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1545
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1546 Stack before: ... anyobject argument
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1547 Stack after: ... anyobject
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1548
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1549 where anyobject may have been mutated, as follows:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1550
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1551 If the object has a __setstate__ method,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1552
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1553 anyobject.__setstate__(argument)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1554
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1555 is called.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1556
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1557 Else the argument must be a dict, the object must have a __dict__, and
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1558 the object is updated via
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1559
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1560 anyobject.__dict__.update(argument)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1561
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1562 This may raise RuntimeError in restricted execution mode (which
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1563 disallows access to __dict__ directly); in that case, the object
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1564 is updated instead via
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1565
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1566 for k, v in argument.items():
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1567 anyobject[k] = v
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1568 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1569
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1570 I(name='INST',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1571 code='i',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1572 arg=stringnl_noescape_pair,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1573 stack_before=[markobject, stackslice],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1574 stack_after=[anyobject],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1575 proto=0,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1576 doc="""Build a class instance.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1577
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1578 This is the protocol 0 version of protocol 1's OBJ opcode.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1579 INST is followed by two newline-terminated strings, giving a
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1580 module and class name, just as for the GLOBAL opcode (and see
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1581 GLOBAL for more details about that). self.find_class(module, name)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1582 is used to get a class object.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1583
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1584 In addition, all the objects on the stack following the topmost
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1585 markobject are gathered into a tuple and popped (along with the
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1586 topmost markobject), just as for the TUPLE opcode.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1587
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1588 Now it gets complicated. If all of these are true:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1589
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1590 + The argtuple is empty (markobject was at the top of the stack
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1591 at the start).
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1592
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1593 + It's an old-style class object (the type of the class object is
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1594 ClassType).
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1595
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1596 + The class object does not have a __getinitargs__ attribute.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1597
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1598 then we want to create an old-style class instance without invoking
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1599 its __init__() method (pickle has waffled on this over the years; not
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1600 calling __init__() is current wisdom). In this case, an instance of
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1601 an old-style dummy class is created, and then we try to rebind its
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1602 __class__ attribute to the desired class object. If this succeeds,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1603 the new instance object is pushed on the stack, and we're done. In
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1604 restricted execution mode it can fail (assignment to __class__ is
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1605 disallowed), and I'm not really sure what happens then -- it looks
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1606 like the code ends up calling the class object's __init__ anyway,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1607 via falling into the next case.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1608
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1609 Else (the argtuple is not empty, it's not an old-style class object,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1610 or the class object does have a __getinitargs__ attribute), the code
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1611 first insists that the class object have a __safe_for_unpickling__
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1612 attribute. Unlike as for the __safe_for_unpickling__ check in REDUCE,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1613 it doesn't matter whether this attribute has a true or false value, it
27513 b1d1fe7fb514 Document the demise of all pretenses of safety, and the difference Guido van Rossum <guido@python.org> parents: 27507 diff changeset	1614 only matters whether it exists (XXX this is a bug; cPickle
b1d1fe7fb514 Document the demise of all pretenses of safety, and the difference Guido van Rossum <guido@python.org> parents: 27507 diff changeset	1615 requires the attribute to be true). If __safe_for_unpickling__
b1d1fe7fb514 Document the demise of all pretenses of safety, and the difference Guido van Rossum <guido@python.org> parents: 27507 diff changeset	1616 doesn't exist, UnpicklingError is raised.
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1617
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1618 Else (the class object does have a __safe_for_unpickling__ attr),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1619 the class object obtained from INST's arguments is applied to the
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1620 argtuple obtained from the stack, and the resulting instance object
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1621 is pushed on the stack.
27535 72b38d5fe61f Updated the INST/OBJ docs, to say they really are (almost) identical Tim Peters <tim.peters@gmail.com> parents: 27532 diff changeset	1622
72b38d5fe61f Updated the INST/OBJ docs, to say they really are (almost) identical Tim Peters <tim.peters@gmail.com> parents: 27532 diff changeset	1623 NOTE: checks for __safe_for_unpickling__ went away in Python 2.3.
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1624 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1625
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1626 I(name='OBJ',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1627 code='o',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1628 arg=None,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1629 stack_before=[markobject, anyobject, stackslice],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1630 stack_after=[anyobject],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1631 proto=1,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1632 doc="""Build a class instance.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1633
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1634 This is the protocol 1 version of protocol 0's INST opcode, and is
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1635 very much like it. The major difference is that the class object
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1636 is taken off the stack, allowing it to be retrieved from the memo
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1637 repeatedly if several instances of the same class are created. This
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1638 can be much more efficient (in both time and space) than repeatedly
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1639 embedding the module and class names in INST opcodes.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1640
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1641 Unlike INST, OBJ takes no arguments from the opcode stream. Instead
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1642 the class object is taken off the stack, immediately above the
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1643 topmost markobject:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1644
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1645 Stack before: ... markobject classobject stackslice
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1646 Stack after: ... new_instance_object
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1647
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1648 As for INST, the remainder of the stack above the markobject is
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1649 gathered into an argument tuple, and then the logic seems identical,
27513 b1d1fe7fb514 Document the demise of all pretenses of safety, and the difference Guido van Rossum <guido@python.org> parents: 27507 diff changeset	1650 except that no __safe_for_unpickling__ check is done (XXX this is
b1d1fe7fb514 Document the demise of all pretenses of safety, and the difference Guido van Rossum <guido@python.org> parents: 27507 diff changeset	1651 a bug; cPickle does test __safe_for_unpickling__). See INST for
b1d1fe7fb514 Document the demise of all pretenses of safety, and the difference Guido van Rossum <guido@python.org> parents: 27507 diff changeset	1652 the gory details.
27535 72b38d5fe61f Updated the INST/OBJ docs, to say they really are (almost) identical Tim Peters <tim.peters@gmail.com> parents: 27532 diff changeset	1653
72b38d5fe61f Updated the INST/OBJ docs, to say they really are (almost) identical Tim Peters <tim.peters@gmail.com> parents: 27532 diff changeset	1654 NOTE: In Python 2.3, INST and OBJ are identical except for how they
72b38d5fe61f Updated the INST/OBJ docs, to say they really are (almost) identical Tim Peters <tim.peters@gmail.com> parents: 27532 diff changeset	1655 get the class object. That was always the intent; the implementations
72b38d5fe61f Updated the INST/OBJ docs, to say they really are (almost) identical Tim Peters <tim.peters@gmail.com> parents: 27532 diff changeset	1656 had diverged for accidental reasons.
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1657 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1658
27462 26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1659 I(name='NEWOBJ',
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1660 code='\x81',
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1661 arg=None,
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1662 stack_before=[anyobject, anyobject],
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1663 stack_after=[anyobject],
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1664 proto=2,
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1665 doc="""Build an object instance.
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1666
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1667 The stack before should be thought of as containing a class
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1668 object followed by an argument tuple (the tuple being the stack
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1669 top). Call these cls and args. They are popped off the stack,
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1670 and the value returned by cls.__new__(cls, *args) is pushed back
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1671 onto the stack.
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1672 """),
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1673
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1674 # Machine control.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1675
27462 26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1676 I(name='PROTO',
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1677 code='\x80',
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1678 arg=uint1,
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1679 stack_before=[],
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1680 stack_after=[],
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1681 proto=2,
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1682 doc="""Protocol version indicator.
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1683
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1684 For protocol 2 and above, a pickle must start with this opcode.
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1685 The argument is the protocol version, an int in range(2, 256).
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1686 """),
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1687
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1688 I(name='STOP',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1689 code='.',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1690 arg=None,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1691 stack_before=[anyobject],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1692 stack_after=[],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1693 proto=0,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1694 doc="""Stop the unpickling machine.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1695
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1696 Every pickle ends with this opcode. The object at the top of the stack
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1697 is popped, and that's the result of unpickling. The stack should be
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1698 empty then.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1699 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1700
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1701 # Ways to deal with persistent IDs.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1702
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1703 I(name='PERSID',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1704 code='P',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1705 arg=stringnl_noescape,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1706 stack_before=[],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1707 stack_after=[anyobject],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1708 proto=0,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1709 doc="""Push an object identified by a persistent ID.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1710
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1711 The pickle module doesn't define what a persistent ID means. PERSID's
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1712 argument is a newline-terminated str-style (no embedded escapes, no
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1713 bracketing quote characters) string, which is "the persistent ID".
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1714 The unpickler passes this string to self.persistent_load(). Whatever
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1715 object that returns is pushed on the stack. There is no implementation
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1716 of persistent_load() in Python's unpickler: it must be supplied by an
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1717 unpickler subclass.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1718 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1719
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1720 I(name='BINPERSID',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1721 code='Q',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1722 arg=None,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1723 stack_before=[anyobject],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1724 stack_after=[anyobject],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1725 proto=1,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1726 doc="""Push an object identified by a persistent ID.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1727
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1728 Like PERSID, except the persistent ID is popped off the stack (instead
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1729 of being a string embedded in the opcode bytestream). The persistent
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1730 ID is passed to self.persistent_load(), and whatever object that
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1731 returns is pushed on the stack. See PERSID for more detail.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1732 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1733 ]
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1734 del I
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1735
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1736 # Verify uniqueness of .name and .code members.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1737 name2i = {}
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1738 code2i = {}
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1739
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1740 for i, d in enumerate(opcodes):
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1741 if d.name in name2i:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1742 raise ValueError("repeated name %r at indices %d and %d" %
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1743 (d.name, name2i[d.name], i))
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1744 if d.code in code2i:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1745 raise ValueError("repeated code %r at indices %d and %d" %
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1746 (d.code, code2i[d.code], i))
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1747
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1748 name2i[d.name] = i
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1749 code2i[d.code] = i
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1750
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1751 del name2i, code2i, i, d
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1752
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1753 ##############################################################################
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1754 # Build a code2op dict, mapping opcode characters to OpcodeInfo records.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1755 # Also ensure we've got the same stuff as pickle.py, although the
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1756 # introspection here is dicey.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1757
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1758 code2op = {}
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1759 for d in opcodes:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1760 code2op[d.code] = d
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1761 del d
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1762
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1763 def assure_pickle_consistency(verbose=False):
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1764 import pickle, re
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1765
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1766 copy = code2op.copy()
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1767 for name in pickle.__all__:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1768 if not re.match("[A-Z][A-Z0-9_]+$", name):
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1769 if verbose:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1770 print "skipping %r: it doesn't look like an opcode name" % name
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1771 continue
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1772 picklecode = getattr(pickle, name)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1773 if not isinstance(picklecode, str) or len(picklecode) != 1:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1774 if verbose:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1775 print ("skipping %r: value %r doesn't look like a pickle "
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1776 "code" % (name, picklecode))
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1777 continue
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1778 if picklecode in copy:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1779 if verbose:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1780 print "checking name %r w/ code %r for consistency" % (
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1781 name, picklecode)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1782 d = copy[picklecode]
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1783 if d.name != name:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1784 raise ValueError("for pickle code %r, pickle.py uses name %r "
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1785 "but we're using name %r" % (picklecode,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1786 name,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1787 d.name))
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1788 # Forget this one. Any left over in copy at the end are a problem
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1789 # of a different kind.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1790 del copy[picklecode]
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1791 else:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1792 raise ValueError("pickle.py appears to have a pickle opcode with "
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1793 "name %r and code %r, but we don't" %
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1794 (name, picklecode))
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1795 if copy:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1796 msg = ["we appear to have pickle opcodes that pickle.py doesn't have:"]
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1797 for code, d in copy.items():
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1798 msg.append(" name %r with code %r" % (d.name, code))
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1799 raise ValueError("\n".join(msg))
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1800
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1801 assure_pickle_consistency()
27506 b480a4f707f6 pickle: Comment repair. Tim Peters <tim.peters@gmail.com> parents: 27473 diff changeset	1802 del assure_pickle_consistency
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1803
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1804 ##############################################################################
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1805 # A pickle opcode generator.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1806
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1807 def genops(pickle):
27424 93ba42cf4e53 Remove a stray quote. Guido van Rossum <guido@python.org> parents: 27423 diff changeset	1808 """Generate all the opcodes in a pickle.
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1809
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1810 'pickle' is a file-like object, or string, containing the pickle.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1811
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1812 Each opcode in the pickle is generated, from the current pickle position,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1813 stopping after a STOP opcode is delivered. A triple is generated for
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1814 each opcode:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1815
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1816 opcode, arg, pos
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1817
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1818 opcode is an OpcodeInfo record, describing the current opcode.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1819
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1820 If the opcode has an argument embedded in the pickle, arg is its decoded
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1821 value, as a Python object. If the opcode doesn't have an argument, arg
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1822 is None.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1823
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1824 If the pickle has a tell() method, pos was the value of pickle.tell()
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1825 before reading the current opcode. If the pickle is a string object,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1826 it's wrapped in a StringIO object, and the latter's tell() result is
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1827 used. Else (the pickle doesn't have a tell(), and it's not obvious how
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1828 to query its current position) pos is None.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1829 """
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1830
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1831 import cStringIO as StringIO
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1832
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1833 if isinstance(pickle, str):
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1834 pickle = StringIO.StringIO(pickle)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1835
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1836 if hasattr(pickle, "tell"):
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1837 getpos = pickle.tell
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1838 else:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1839 getpos = lambda: None
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1840
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1841 while True:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1842 pos = getpos()
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1843 code = pickle.read(1)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1844 opcode = code2op.get(code)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1845 if opcode is None:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1846 if code == "":
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1847 raise ValueError("pickle exhausted before seeing STOP")
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1848 else:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1849 raise ValueError("at position %s, opcode %r unknown" % (
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1850 pos is None and "<unknown>" or pos,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1851 code))
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1852 if opcode.arg is None:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1853 arg = None
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1854 else:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1855 arg = opcode.arg.reader(pickle)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1856 yield opcode, arg, pos
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1857 if code == '.':
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1858 assert opcode.name == 'STOP'
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1859 break
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1860
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1861 ##############################################################################
45273 c7f9f9fdcf82 Complete an open todo on pickletools -- add a pickle optimizer. Raymond Hettinger <python@rcn.com> parents: 34590 diff changeset	1862 # A pickle optimizer.
c7f9f9fdcf82 Complete an open todo on pickletools -- add a pickle optimizer. Raymond Hettinger <python@rcn.com> parents: 34590 diff changeset	1863
c7f9f9fdcf82 Complete an open todo on pickletools -- add a pickle optimizer. Raymond Hettinger <python@rcn.com> parents: 34590 diff changeset	1864 def optimize(p):
c7f9f9fdcf82 Complete an open todo on pickletools -- add a pickle optimizer. Raymond Hettinger <python@rcn.com> parents: 34590 diff changeset	1865 'Optimize a pickle string by removing unused PUT opcodes'
c7f9f9fdcf82 Complete an open todo on pickletools -- add a pickle optimizer. Raymond Hettinger <python@rcn.com> parents: 34590 diff changeset	1866 gets = set() # set of args used by a GET opcode
c7f9f9fdcf82 Complete an open todo on pickletools -- add a pickle optimizer. Raymond Hettinger <python@rcn.com> parents: 34590 diff changeset	1867 puts = [] # (arg, startpos, stoppos) for the PUT opcodes
c7f9f9fdcf82 Complete an open todo on pickletools -- add a pickle optimizer. Raymond Hettinger <python@rcn.com> parents: 34590 diff changeset	1868 prevpos = None # set to pos if previous opcode was a PUT
c7f9f9fdcf82 Complete an open todo on pickletools -- add a pickle optimizer. Raymond Hettinger <python@rcn.com> parents: 34590 diff changeset	1869 for opcode, arg, pos in genops(p):
c7f9f9fdcf82 Complete an open todo on pickletools -- add a pickle optimizer. Raymond Hettinger <python@rcn.com> parents: 34590 diff changeset	1870 if prevpos is not None:
c7f9f9fdcf82 Complete an open todo on pickletools -- add a pickle optimizer. Raymond Hettinger <python@rcn.com> parents: 34590 diff changeset	1871 puts.append((prevarg, prevpos, pos))
c7f9f9fdcf82 Complete an open todo on pickletools -- add a pickle optimizer. Raymond Hettinger <python@rcn.com> parents: 34590 diff changeset	1872 prevpos = None
c7f9f9fdcf82 Complete an open todo on pickletools -- add a pickle optimizer. Raymond Hettinger <python@rcn.com> parents: 34590 diff changeset	1873 if 'PUT' in opcode.name:
c7f9f9fdcf82 Complete an open todo on pickletools -- add a pickle optimizer. Raymond Hettinger <python@rcn.com> parents: 34590 diff changeset	1874 prevarg, prevpos = arg, pos
c7f9f9fdcf82 Complete an open todo on pickletools -- add a pickle optimizer. Raymond Hettinger <python@rcn.com> parents: 34590 diff changeset	1875 elif 'GET' in opcode.name:
c7f9f9fdcf82 Complete an open todo on pickletools -- add a pickle optimizer. Raymond Hettinger <python@rcn.com> parents: 34590 diff changeset	1876 gets.add(arg)
c7f9f9fdcf82 Complete an open todo on pickletools -- add a pickle optimizer. Raymond Hettinger <python@rcn.com> parents: 34590 diff changeset	1877
c7f9f9fdcf82 Complete an open todo on pickletools -- add a pickle optimizer. Raymond Hettinger <python@rcn.com> parents: 34590 diff changeset	1878 # Copy the pickle string except for PUTS without a corresponding GET
c7f9f9fdcf82 Complete an open todo on pickletools -- add a pickle optimizer. Raymond Hettinger <python@rcn.com> parents: 34590 diff changeset	1879 s = []
c7f9f9fdcf82 Complete an open todo on pickletools -- add a pickle optimizer. Raymond Hettinger <python@rcn.com> parents: 34590 diff changeset	1880 i = 0
c7f9f9fdcf82 Complete an open todo on pickletools -- add a pickle optimizer. Raymond Hettinger <python@rcn.com> parents: 34590 diff changeset	1881 for arg, start, stop in puts:
c7f9f9fdcf82 Complete an open todo on pickletools -- add a pickle optimizer. Raymond Hettinger <python@rcn.com> parents: 34590 diff changeset	1882 j = stop if (arg in gets) else start
c7f9f9fdcf82 Complete an open todo on pickletools -- add a pickle optimizer. Raymond Hettinger <python@rcn.com> parents: 34590 diff changeset	1883 s.append(p[i:j])
c7f9f9fdcf82 Complete an open todo on pickletools -- add a pickle optimizer. Raymond Hettinger <python@rcn.com> parents: 34590 diff changeset	1884 i = stop
c7f9f9fdcf82 Complete an open todo on pickletools -- add a pickle optimizer. Raymond Hettinger <python@rcn.com> parents: 34590 diff changeset	1885 s.append(p[i:])
c7f9f9fdcf82 Complete an open todo on pickletools -- add a pickle optimizer. Raymond Hettinger <python@rcn.com> parents: 34590 diff changeset	1886 return ''.join(s)
c7f9f9fdcf82 Complete an open todo on pickletools -- add a pickle optimizer. Raymond Hettinger <python@rcn.com> parents: 34590 diff changeset	1887
c7f9f9fdcf82 Complete an open todo on pickletools -- add a pickle optimizer. Raymond Hettinger <python@rcn.com> parents: 34590 diff changeset	1888 ##############################################################################
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1889 # A symbolic pickle disassembler.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1890
27703 0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	1891 def dis(pickle, out=None, memo=None, indentlevel=4):
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1892 """Produce a symbolic disassembly of a pickle.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1893
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1894 'pickle' is a file-like object, or string, containing a (at least one)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1895 pickle. The pickle is disassembled from the current position, through
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1896 the first STOP opcode encountered.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1897
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1898 Optional arg 'out' is a file-like object to which the disassembly is
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1899 printed. It defaults to sys.stdout.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1900
27703 0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	1901 Optional arg 'memo' is a Python dict, used as the pickle's memo. It
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	1902 may be mutated by dis(), if the pickle contains PUT or BINPUT opcodes.
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	1903 Passing the same memo object to another dis() call then allows disassembly
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	1904 to proceed across multiple pickles that were all created by the same
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	1905 pickler with the same memo. Ordinarily you don't need to worry about this.
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	1906
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1907 Optional arg indentlevel is the number of blanks by which to indent
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1908 a new MARK level. It defaults to 4.
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1909
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1910 In addition to printing the disassembly, some sanity checks are made:
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1911
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1912 + All embedded opcode arguments "make sense".
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1913
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1914 + Explicit and implicit pop operations have enough items on the stack.
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1915
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1916 + When an opcode implicitly refers to a markobject, a markobject is
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1917 actually on the stack.
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1918
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1919 + A memo entry isn't referenced before it's defined.
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1920
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1921 + The markobject isn't stored in the memo.
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1922
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1923 + A memo entry isn't redefined.
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1924 """
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1925
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1926 # Most of the hair here is for sanity checks, but most of it is needed
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1927 # anyway to detect when a protocol 0 POP takes a MARK off the stack
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1928 # (which in turn is needed to indent MARK blocks correctly).
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1929
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1930 stack = [] # crude emulation of unpickler stack
27703 0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	1931 if memo is None:
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	1932 memo = {} # crude emulation of unpicker memo
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1933 maxproto = -1 # max protocol number seen
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1934 markstack = [] # bytecode positions of MARK opcodes
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1935 indentchunk = ' ' * indentlevel
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1936 errormsg = None
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1937 for opcode, arg, pos in genops(pickle):
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1938 if pos is not None:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1939 print >> out, "%5d:" % pos,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1940
27471 05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	1941 line = "%-4s %s%s" % (repr(opcode.code)[1:-1],
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	1942 indentchunk * len(markstack),
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	1943 opcode.name)
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1944
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1945 maxproto = max(maxproto, opcode.proto)
27532 02f138c639a0 dis(): Simplified stack emulation a bit. Tim Peters <tim.peters@gmail.com> parents: 27522 diff changeset	1946 before = opcode.stack_before # don't mutate
02f138c639a0 dis(): Simplified stack emulation a bit. Tim Peters <tim.peters@gmail.com> parents: 27522 diff changeset	1947 after = opcode.stack_after # don't mutate
02f138c639a0 dis(): Simplified stack emulation a bit. Tim Peters <tim.peters@gmail.com> parents: 27522 diff changeset	1948 numtopop = len(before)
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1949
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1950 # See whether a MARK should be popped.
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1951 markmsg = None
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1952 if markobject in before or (opcode.name == "POP" and
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1953 stack and
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1954 stack[-1] is markobject):
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1955 assert markobject not in after
27532 02f138c639a0 dis(): Simplified stack emulation a bit. Tim Peters <tim.peters@gmail.com> parents: 27522 diff changeset	1956 if __debug__:
02f138c639a0 dis(): Simplified stack emulation a bit. Tim Peters <tim.peters@gmail.com> parents: 27522 diff changeset	1957 if markobject in before:
02f138c639a0 dis(): Simplified stack emulation a bit. Tim Peters <tim.peters@gmail.com> parents: 27522 diff changeset	1958 assert before[-1] is stackslice
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1959 if markstack:
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1960 markpos = markstack.pop()
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1961 if markpos is None:
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1962 markmsg = "(MARK at unknown opcode offset)"
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1963 else:
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1964 markmsg = "(MARK at %d)" % markpos
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1965 # Pop everything at and after the topmost markobject.
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1966 while stack[-1] is not markobject:
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1967 stack.pop()
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1968 stack.pop()
27532 02f138c639a0 dis(): Simplified stack emulation a bit. Tim Peters <tim.peters@gmail.com> parents: 27522 diff changeset	1969 # Stop later code from popping too much.
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1970 try:
27532 02f138c639a0 dis(): Simplified stack emulation a bit. Tim Peters <tim.peters@gmail.com> parents: 27522 diff changeset	1971 numtopop = before.index(markobject)
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1972 except ValueError:
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1973 assert opcode.name == "POP"
27532 02f138c639a0 dis(): Simplified stack emulation a bit. Tim Peters <tim.peters@gmail.com> parents: 27522 diff changeset	1974 numtopop = 0
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1975 else:
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1976 errormsg = markmsg = "no MARK exists on stack"
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1977
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1978 # Check for correct memo usage.
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1979 if opcode.name in ("PUT", "BINPUT", "LONG_BINPUT"):
27532 02f138c639a0 dis(): Simplified stack emulation a bit. Tim Peters <tim.peters@gmail.com> parents: 27522 diff changeset	1980 assert arg is not None
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1981 if arg in memo:
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1982 errormsg = "memo key %r already defined" % arg
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1983 elif not stack:
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1984 errormsg = "stack is empty -- can't store into memo"
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1985 elif stack[-1] is markobject:
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1986 errormsg = "can't store markobject in the memo"
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1987 else:
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1988 memo[arg] = stack[-1]
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1989
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1990 elif opcode.name in ("GET", "BINGET", "LONG_BINGET"):
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1991 if arg in memo:
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1992 assert len(after) == 1
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1993 after = [memo[arg]] # for better stack emulation
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1994 else:
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1995 errormsg = "memo key %r has never been stored into" % arg
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1996
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1997 if arg is not None or markmsg:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1998 # make a mild effort to align arguments
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1999 line += ' ' * (10 - len(opcode.name))
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2000 if arg is not None:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2001 line += ' ' + repr(arg)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2002 if markmsg:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2003 line += ' ' + markmsg
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2004 print >> out, line
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2005
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	2006 if errormsg:
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	2007 # Note that we delayed complaining until the offending opcode
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	2008 # was printed.
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	2009 raise ValueError(errormsg)
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	2010
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	2011 # Emulate the stack effects.
27532 02f138c639a0 dis(): Simplified stack emulation a bit. Tim Peters <tim.peters@gmail.com> parents: 27522 diff changeset	2012 if len(stack) < numtopop:
02f138c639a0 dis(): Simplified stack emulation a bit. Tim Peters <tim.peters@gmail.com> parents: 27522 diff changeset	2013 raise ValueError("tries to pop %d items from stack with "
02f138c639a0 dis(): Simplified stack emulation a bit. Tim Peters <tim.peters@gmail.com> parents: 27522 diff changeset	2014 "only %d items" % (numtopop, len(stack)))
02f138c639a0 dis(): Simplified stack emulation a bit. Tim Peters <tim.peters@gmail.com> parents: 27522 diff changeset	2015 if numtopop:
02f138c639a0 dis(): Simplified stack emulation a bit. Tim Peters <tim.peters@gmail.com> parents: 27522 diff changeset	2016 del stack[-numtopop:]
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	2017 if markobject in after:
27532 02f138c639a0 dis(): Simplified stack emulation a bit. Tim Peters <tim.peters@gmail.com> parents: 27522 diff changeset	2018 assert markobject not in before
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2019 markstack.append(pos)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2020
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	2021 stack.extend(after)
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	2022
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	2023 print >> out, "highest protocol among opcodes =", maxproto
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	2024 if stack:
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	2025 raise ValueError("stack not empty after STOP: %r" % stack)
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2026
34590 ff3def77b129 An instance of class PicklingError was used here simply as an example of Tim Peters <tim.peters@gmail.com> parents: 34128 diff changeset	2027 # For use in the doctest, simply as an example of a class to pickle.
ff3def77b129 An instance of class PicklingError was used here simply as an example of Tim Peters <tim.peters@gmail.com> parents: 34128 diff changeset	2028 class _Example:
ff3def77b129 An instance of class PicklingError was used here simply as an example of Tim Peters <tim.peters@gmail.com> parents: 34128 diff changeset	2029 def __init__(self, value):
ff3def77b129 An instance of class PicklingError was used here simply as an example of Tim Peters <tim.peters@gmail.com> parents: 34128 diff changeset	2030 self.value = value
ff3def77b129 An instance of class PicklingError was used here simply as an example of Tim Peters <tim.peters@gmail.com> parents: 34128 diff changeset	2031
27472 de4061584c83 _dis_test should be a raw string now that it contains \x escapes. Guido van Rossum <guido@python.org> parents: 27471 diff changeset	2032 _dis_test = r"""
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2033 >>> import pickle
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2034 >>> x = [1, 2, (3, 4), {'abc': u"def"}]
27468 6ffafcef98fb Fix one disassembly output now that empty tuples are no longer Guido van Rossum <guido@python.org> parents: 27463 diff changeset	2035 >>> pkl = pickle.dumps(x, 0)
6ffafcef98fb Fix one disassembly output now that empty tuples are no longer Guido van Rossum <guido@python.org> parents: 27463 diff changeset	2036 >>> dis(pkl)
27471 05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2037 0: ( MARK
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2038 1: l LIST (MARK at 0)
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2039 2: p PUT 0
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2040 5: I INT 1
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2041 8: a APPEND
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2042 9: I INT 2
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2043 12: a APPEND
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2044 13: ( MARK
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2045 14: I INT 3
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2046 17: I INT 4
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2047 20: t TUPLE (MARK at 13)
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2048 21: p PUT 1
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2049 24: a APPEND
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2050 25: ( MARK
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2051 26: d DICT (MARK at 25)
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2052 27: p PUT 2
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2053 30: S STRING 'abc'
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2054 37: p PUT 3
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2055 40: V UNICODE u'def'
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2056 45: p PUT 4
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2057 48: s SETITEM
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2058 49: a APPEND
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2059 50: . STOP
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	2060 highest protocol among opcodes = 0
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2061
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2062 Try again with a "binary" pickle.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2063
27468 6ffafcef98fb Fix one disassembly output now that empty tuples are no longer Guido van Rossum <guido@python.org> parents: 27463 diff changeset	2064 >>> pkl = pickle.dumps(x, 1)
6ffafcef98fb Fix one disassembly output now that empty tuples are no longer Guido van Rossum <guido@python.org> parents: 27463 diff changeset	2065 >>> dis(pkl)
27471 05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2066 0: ] EMPTY_LIST
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2067 1: q BINPUT 0
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2068 3: ( MARK
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2069 4: K BININT1 1
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2070 6: K BININT1 2
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2071 8: ( MARK
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2072 9: K BININT1 3
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2073 11: K BININT1 4
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2074 13: t TUPLE (MARK at 8)
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2075 14: q BINPUT 1
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2076 16: } EMPTY_DICT
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2077 17: q BINPUT 2
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2078 19: U SHORT_BINSTRING 'abc'
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2079 24: q BINPUT 3
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2080 26: X BINUNICODE u'def'
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2081 34: q BINPUT 4
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2082 36: s SETITEM
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2083 37: e APPENDS (MARK at 3)
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2084 38: . STOP
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	2085 highest protocol among opcodes = 1
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2086
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2087 Exercise the INST/OBJ/BUILD family.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2088
51020 0139648526d9 Merged revisions 68903,68906 via svnmerge from Mark Dickinson <dickinsm@gmail.com> parents: 47189 diff changeset	2089 >>> import pickletools
0139648526d9 Merged revisions 68903,68906 via svnmerge from Mark Dickinson <dickinsm@gmail.com> parents: 47189 diff changeset	2090 >>> dis(pickle.dumps(pickletools.dis, 0))
0139648526d9 Merged revisions 68903,68906 via svnmerge from Mark Dickinson <dickinsm@gmail.com> parents: 47189 diff changeset	2091 0: c GLOBAL 'pickletools dis'
0139648526d9 Merged revisions 68903,68906 via svnmerge from Mark Dickinson <dickinsm@gmail.com> parents: 47189 diff changeset	2092 17: p PUT 0
0139648526d9 Merged revisions 68903,68906 via svnmerge from Mark Dickinson <dickinsm@gmail.com> parents: 47189 diff changeset	2093 20: . STOP
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	2094 highest protocol among opcodes = 0
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2095
34590 ff3def77b129 An instance of class PicklingError was used here simply as an example of Tim Peters <tim.peters@gmail.com> parents: 34128 diff changeset	2096 >>> from pickletools import _Example
ff3def77b129 An instance of class PicklingError was used here simply as an example of Tim Peters <tim.peters@gmail.com> parents: 34128 diff changeset	2097 >>> x = [_Example(42)] * 2
27431 f8a90085acc4 Begin the change from 'binary vs. text mode' to 'protocol 0, 1, 2'. Guido van Rossum <guido@python.org> parents: 27429 diff changeset	2098 >>> dis(pickle.dumps(x, 0))
27471 05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2099 0: ( MARK
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2100 1: l LIST (MARK at 0)
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2101 2: p PUT 0
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2102 5: ( MARK
34590 ff3def77b129 An instance of class PicklingError was used here simply as an example of Tim Peters <tim.peters@gmail.com> parents: 34128 diff changeset	2103 6: i INST 'pickletools _Example' (MARK at 5)
27471 05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2104 28: p PUT 1
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2105 31: ( MARK
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2106 32: d DICT (MARK at 31)
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2107 33: p PUT 2
34590 ff3def77b129 An instance of class PicklingError was used here simply as an example of Tim Peters <tim.peters@gmail.com> parents: 34128 diff changeset	2108 36: S STRING 'value'
ff3def77b129 An instance of class PicklingError was used here simply as an example of Tim Peters <tim.peters@gmail.com> parents: 34128 diff changeset	2109 45: p PUT 3
ff3def77b129 An instance of class PicklingError was used here simply as an example of Tim Peters <tim.peters@gmail.com> parents: 34128 diff changeset	2110 48: I INT 42
ff3def77b129 An instance of class PicklingError was used here simply as an example of Tim Peters <tim.peters@gmail.com> parents: 34128 diff changeset	2111 52: s SETITEM
ff3def77b129 An instance of class PicklingError was used here simply as an example of Tim Peters <tim.peters@gmail.com> parents: 34128 diff changeset	2112 53: b BUILD
ff3def77b129 An instance of class PicklingError was used here simply as an example of Tim Peters <tim.peters@gmail.com> parents: 34128 diff changeset	2113 54: a APPEND
ff3def77b129 An instance of class PicklingError was used here simply as an example of Tim Peters <tim.peters@gmail.com> parents: 34128 diff changeset	2114 55: g GET 1
ff3def77b129 An instance of class PicklingError was used here simply as an example of Tim Peters <tim.peters@gmail.com> parents: 34128 diff changeset	2115 58: a APPEND
ff3def77b129 An instance of class PicklingError was used here simply as an example of Tim Peters <tim.peters@gmail.com> parents: 34128 diff changeset	2116 59: . STOP
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	2117 highest protocol among opcodes = 0
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2118
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2119 >>> dis(pickle.dumps(x, 1))
27471 05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2120 0: ] EMPTY_LIST
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2121 1: q BINPUT 0
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2122 3: ( MARK
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2123 4: ( MARK
34590 ff3def77b129 An instance of class PicklingError was used here simply as an example of Tim Peters <tim.peters@gmail.com> parents: 34128 diff changeset	2124 5: c GLOBAL 'pickletools _Example'
27471 05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2125 27: q BINPUT 1
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2126 29: o OBJ (MARK at 4)
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2127 30: q BINPUT 2
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2128 32: } EMPTY_DICT
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2129 33: q BINPUT 3
34590 ff3def77b129 An instance of class PicklingError was used here simply as an example of Tim Peters <tim.peters@gmail.com> parents: 34128 diff changeset	2130 35: U SHORT_BINSTRING 'value'
ff3def77b129 An instance of class PicklingError was used here simply as an example of Tim Peters <tim.peters@gmail.com> parents: 34128 diff changeset	2131 42: q BINPUT 4
ff3def77b129 An instance of class PicklingError was used here simply as an example of Tim Peters <tim.peters@gmail.com> parents: 34128 diff changeset	2132 44: K BININT1 42
ff3def77b129 An instance of class PicklingError was used here simply as an example of Tim Peters <tim.peters@gmail.com> parents: 34128 diff changeset	2133 46: s SETITEM
ff3def77b129 An instance of class PicklingError was used here simply as an example of Tim Peters <tim.peters@gmail.com> parents: 34128 diff changeset	2134 47: b BUILD
ff3def77b129 An instance of class PicklingError was used here simply as an example of Tim Peters <tim.peters@gmail.com> parents: 34128 diff changeset	2135 48: h BINGET 2
ff3def77b129 An instance of class PicklingError was used here simply as an example of Tim Peters <tim.peters@gmail.com> parents: 34128 diff changeset	2136 50: e APPENDS (MARK at 3)
ff3def77b129 An instance of class PicklingError was used here simply as an example of Tim Peters <tim.peters@gmail.com> parents: 34128 diff changeset	2137 51: . STOP
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	2138 highest protocol among opcodes = 1
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2139
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2140 Try "the canonical" recursive-object test.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2141
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2142 >>> L = []
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2143 >>> T = L,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2144 >>> L.append(T)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2145 >>> L[0] is T
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2146 True
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2147 >>> T[0] is L
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2148 True
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2149 >>> L[0][0] is L
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2150 True
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2151 >>> T[0][0] is T
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2152 True
27431 f8a90085acc4 Begin the change from 'binary vs. text mode' to 'protocol 0, 1, 2'. Guido van Rossum <guido@python.org> parents: 27429 diff changeset	2153 >>> dis(pickle.dumps(L, 0))
27471 05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2154 0: ( MARK
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2155 1: l LIST (MARK at 0)
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2156 2: p PUT 0
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2157 5: ( MARK
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2158 6: g GET 0
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2159 9: t TUPLE (MARK at 5)
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2160 10: p PUT 1
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2161 13: a APPEND
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2162 14: . STOP
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	2163 highest protocol among opcodes = 0
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	2164
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2165 >>> dis(pickle.dumps(L, 1))
27471 05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2166 0: ] EMPTY_LIST
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2167 1: q BINPUT 0
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2168 3: ( MARK
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2169 4: h BINGET 0
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2170 6: t TUPLE (MARK at 3)
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2171 7: q BINPUT 1
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2172 9: a APPEND
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2173 10: . STOP
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	2174 highest protocol among opcodes = 1
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2175
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	2176 Note that, in the protocol 0 pickle of the recursive tuple, the disassembler
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	2177 has to emulate the stack in order to realize that the POP opcode at 16 gets
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	2178 rid of the MARK at 0.
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2179
27431 f8a90085acc4 Begin the change from 'binary vs. text mode' to 'protocol 0, 1, 2'. Guido van Rossum <guido@python.org> parents: 27429 diff changeset	2180 >>> dis(pickle.dumps(T, 0))
27471 05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2181 0: ( MARK
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2182 1: ( MARK
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2183 2: l LIST (MARK at 1)
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2184 3: p PUT 0
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2185 6: ( MARK
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2186 7: g GET 0
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2187 10: t TUPLE (MARK at 6)
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2188 11: p PUT 1
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2189 14: a APPEND
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2190 15: 0 POP
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	2191 16: 0 POP (MARK at 0)
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	2192 17: g GET 1
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	2193 20: . STOP
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	2194 highest protocol among opcodes = 0
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	2195
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2196 >>> dis(pickle.dumps(T, 1))
27471 05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2197 0: ( MARK
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2198 1: ] EMPTY_LIST
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2199 2: q BINPUT 0
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2200 4: ( MARK
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2201 5: h BINGET 0
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2202 7: t TUPLE (MARK at 4)
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2203 8: q BINPUT 1
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2204 10: a APPEND
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2205 11: 1 POP_MARK (MARK at 0)
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2206 12: h BINGET 1
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2207 14: . STOP
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	2208 highest protocol among opcodes = 1
27471 05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2209
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2210 Try protocol 2.
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2211
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2212 >>> dis(pickle.dumps(L, 2))
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2213 0: \x80 PROTO 2
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2214 2: ] EMPTY_LIST
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2215 3: q BINPUT 0
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2216 5: h BINGET 0
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2217 7: \x85 TUPLE1
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2218 8: q BINPUT 1
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2219 10: a APPEND
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2220 11: . STOP
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	2221 highest protocol among opcodes = 2
27471 05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2222
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2223 >>> dis(pickle.dumps(T, 2))
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2224 0: \x80 PROTO 2
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2225 2: ] EMPTY_LIST
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2226 3: q BINPUT 0
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2227 5: h BINGET 0
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2228 7: \x85 TUPLE1
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2229 8: q BINPUT 1
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2230 10: a APPEND
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2231 11: 0 POP
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2232 12: h BINGET 1
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2233 14: . STOP
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	2234 highest protocol among opcodes = 2
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2235 """
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2236
27703 0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	2237 _memo_test = r"""
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	2238 >>> import pickle
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	2239 >>> from StringIO import StringIO
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	2240 >>> f = StringIO()
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	2241 >>> p = pickle.Pickler(f, 2)
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	2242 >>> x = [1, 2, 3]
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	2243 >>> p.dump(x)
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	2244 >>> p.dump(x)
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	2245 >>> f.seek(0)
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	2246 >>> memo = {}
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	2247 >>> dis(f, memo=memo)
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	2248 0: \x80 PROTO 2
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	2249 2: ] EMPTY_LIST
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	2250 3: q BINPUT 0
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	2251 5: ( MARK
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	2252 6: K BININT1 1
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	2253 8: K BININT1 2
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	2254 10: K BININT1 3
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	2255 12: e APPENDS (MARK at 5)
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	2256 13: . STOP
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	2257 highest protocol among opcodes = 2
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	2258 >>> dis(f, memo=memo)
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	2259 14: \x80 PROTO 2
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	2260 16: h BINGET 0
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	2261 18: . STOP
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	2262 highest protocol among opcodes = 2
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	2263 """
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	2264
27468 6ffafcef98fb Fix one disassembly output now that empty tuples are no longer Guido van Rossum <guido@python.org> parents: 27463 diff changeset	2265 __test__ = {'disassembler_test': _dis_test,
27703 0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	2266 'disassembler_memo_test': _memo_test,
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2267 }
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2268
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2269 def _test():
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2270 import doctest
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2271 return doctest.testmod()
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2272
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2273 if __name__ == "__main__":
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2274 _test()

Mercurial > cpython-lru_cache-weakref

annotate Lib/pickletools.py @ 80661:94de77bd0b4b 2.7