cpython-lru_cache-weakref: Lib/pickletools.py annotate

annotate Lib/pickletools.py @ 27998:e27f58a50737 legacy-trunk

SF bug #690377: Dud footnote reference in distutils docs

author	Raymond Hettinger <python@rcn.com>
date	Fri, 21 Feb 2003 02:47:09 +0000
parents	0428d59069a9
children	867fef2b5060

rev	line source
27518 b2b574118ffd minor grammar tweaks Skip Montanaro <skip@pobox.com> parents: 27513 diff changeset	1 '''"Executable documentation" for the pickle module.
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	3 Extensive comments about the pickle protocols and pickle-machine opcodes
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	4 can be found here. Some functions meant for external use:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	5
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	6 genops(pickle)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	7 Generate all the opcodes in a pickle, as (opcode, arg, position) triples.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	8
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	9 dis(pickle, out=None, indentlevel=4)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	10 Print a symbolic disassembly of a pickle.
27518 b2b574118ffd minor grammar tweaks Skip Montanaro <skip@pobox.com> parents: 27513 diff changeset	11 '''
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	12
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	13 # Other ideas:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	14 #
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	15 # - A pickle verifier: read a pickle and check it exhaustively for
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	16 # well-formedness. dis() does a lot of this already.
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	17 #
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	18 # - A protocol identifier: examine a pickle and return its protocol number
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	19 # (== the highest .proto attr value among all the opcodes in the pickle).
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	20 # dis() already prints this info at the end.
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	21 #
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	22 # - A pickle optimizer: for example, tuple-building code is sometimes more
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	23 # elaborate than necessary, catering for the possibility that the tuple
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	24 # is recursive. Or lots of times a PUT is generated that's never accessed
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	25 # by a later GET.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	26
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	27
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	28 """
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	29 "A pickle" is a program for a virtual pickle machine (PM, but more accurately
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	30 called an unpickling machine). It's a sequence of opcodes, interpreted by the
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	31 PM, building an arbitrarily complex Python object.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	32
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	33 For the most part, the PM is very simple: there are no looping, testing, or
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	34 conditional instructions, no arithmetic and no function calls. Opcodes are
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	35 executed once each, from first to last, until a STOP opcode is reached.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	36
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	37 The PM has two data areas, "the stack" and "the memo".
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	38
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	39 Many opcodes push Python objects onto the stack; e.g., INT pushes a Python
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	40 integer object on the stack, whose value is gotten from a decimal string
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	41 literal immediately following the INT opcode in the pickle bytestream. Other
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	42 opcodes take Python objects off the stack. The result of unpickling is
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	43 whatever object is left on the stack when the final STOP opcode is executed.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	44
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	45 The memo is simply an array of objects, or it can be implemented as a dict
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	46 mapping little integers to objects. The memo serves as the PM's "long term
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	47 memory", and the little integers indexing the memo are akin to variable
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	48 names. Some opcodes pop a stack object into the memo at a given index,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	49 and others push a memo object at a given index onto the stack again.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	50
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	51 At heart, that's all the PM has. Subtleties arise for these reasons:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	52
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	53 + Object identity. Objects can be arbitrarily complex, and subobjects
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	54 may be shared (for example, the list [a, a] refers to the same object a
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	55 twice). It can be vital that unpickling recreate an isomorphic object
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	56 graph, faithfully reproducing sharing.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	57
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	58 + Recursive objects. For example, after "L = []; L.append(L)", L is a
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	59 list, and L[0] is the same list. This is related to the object identity
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	60 point, and some sequences of pickle opcodes are subtle in order to
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	61 get the right result in all cases.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	62
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	63 + Things pickle doesn't know everything about. Examples of things pickle
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	64 does know everything about are Python's builtin scalar and container
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	65 types, like ints and tuples. They generally have opcodes dedicated to
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	66 them. For things like module references and instances of user-defined
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	67 classes, pickle's knowledge is limited. Historically, many enhancements
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	68 have been made to the pickle protocol in order to do a better (faster,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	69 and/or more compact) job on those.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	70
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	71 + Backward compatibility and micro-optimization. As explained below,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	72 pickle opcodes never go away, not even when better ways to do a thing
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	73 get invented. The repertoire of the PM just keeps growing over time.
27462 26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	74 For example, protocol 0 had two opcodes for building Python integers (INT
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	75 and LONG), protocol 1 added three more for more-efficient pickling of short
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	76 integers, and protocol 2 added two more for more-efficient pickling of
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	77 long integers (before protocol 2, the only ways to pickle a Python long
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	78 took time quadratic in the number of digits, for both pickling and
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	79 unpickling). "Opcode bloat" isn't so much a subtlety as a source of
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	80 wearying complication.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	81
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	82
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	83 Pickle protocols:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	84
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	85 For compatibility, the meaning of a pickle opcode never changes. Instead new
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	86 pickle opcodes get added, and each version's unpickler can handle all the
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	87 pickle opcodes in all protocol versions to date. So old pickles continue to
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	88 be readable forever. The pickler can generally be told to restrict itself to
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	89 the subset of opcodes available under previous protocol versions too, so that
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	90 users can create pickles under the current version readable by older
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	91 versions. However, a pickle does not contain its version number embedded
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	92 within it. If an older unpickler tries to read a pickle using a later
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	93 protocol, the result is most likely an exception due to seeing an unknown (in
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	94 the older unpickler) opcode.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	95
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	96 The original pickle used what's now called "protocol 0", and what was called
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	97 "text mode" before Python 2.3. The entire pickle bytestream is made up of
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	98 printable 7-bit ASCII characters, plus the newline character, in protocol 0.
27462 26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	99 That's why it was called text mode. Protocol 0 is small and elegant, but
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	100 sometimes painfully inefficient.
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	101
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	102 The second major set of additions is now called "protocol 1", and was called
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	103 "binary mode" before Python 2.3. This added many opcodes with arguments
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	104 consisting of arbitrary bytes, including NUL bytes and unprintable "high bit"
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	105 bytes. Binary mode pickles can be substantially smaller than equivalent
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	106 text mode pickles, and sometimes faster too; e.g., BININT represents a 4-byte
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	107 int as 4 bytes following the opcode, which is cheaper to unpickle than the
27462 26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	108 (perhaps) 11-character decimal string attached to INT. Protocol 1 also added
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	109 a number of opcodes that operate on many stack elements at once (like APPENDS
27463 2ee9742aadd8 Comments. Tim Peters <tim.peters@gmail.com> parents: 27462 diff changeset	110 and SETITEMS), and "shortcut" opcodes (like EMPTY_DICT and EMPTY_TUPLE).
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	111
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	112 The third major set of additions came in Python 2.3, and is called "protocol
27462 26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	113 2". This added:
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	114
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	115 - A better way to pickle instances of new-style classes (NEWOBJ).
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	116
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	117 - A way for a pickle to identify its protocol (PROTO).
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	118
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	119 - Time- and space- efficient pickling of long ints (LONG{1,4}).
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	120
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	121 - Shortcuts for small tuples (TUPLE{1,2,3}}.
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	122
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	123 - Dedicated opcodes for bools (NEWTRUE, NEWFALSE).
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	124
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	125 - The "extension registry", a vector of popular objects that can be pushed
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	126 efficiently by index (EXT{1,2,4}). This is akin to the memo and GET, but
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	127 the registry contents are predefined (there's nothing akin to the memo's
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	128 PUT).
27513 b1d1fe7fb514 Document the demise of all pretenses of safety, and the difference Guido van Rossum <guido@python.org> parents: 27507 diff changeset	129
27518 b2b574118ffd minor grammar tweaks Skip Montanaro <skip@pobox.com> parents: 27513 diff changeset	130 Another independent change with Python 2.3 is the abandonment of any
b2b574118ffd minor grammar tweaks Skip Montanaro <skip@pobox.com> parents: 27513 diff changeset	131 pretense that it might be safe to load pickles received from untrusted
27513 b1d1fe7fb514 Document the demise of all pretenses of safety, and the difference Guido van Rossum <guido@python.org> parents: 27507 diff changeset	132 parties -- no sufficient security analysis has been done to guarantee
27518 b2b574118ffd minor grammar tweaks Skip Montanaro <skip@pobox.com> parents: 27513 diff changeset	133 this and there isn't a use case that warrants the expense of such an
27513 b1d1fe7fb514 Document the demise of all pretenses of safety, and the difference Guido van Rossum <guido@python.org> parents: 27507 diff changeset	134 analysis.
b1d1fe7fb514 Document the demise of all pretenses of safety, and the difference Guido van Rossum <guido@python.org> parents: 27507 diff changeset	135
b1d1fe7fb514 Document the demise of all pretenses of safety, and the difference Guido van Rossum <guido@python.org> parents: 27507 diff changeset	136 To this end, all tests for __safe_for_unpickling__ or for
b1d1fe7fb514 Document the demise of all pretenses of safety, and the difference Guido van Rossum <guido@python.org> parents: 27507 diff changeset	137 copy_reg.safe_constructors are removed from the unpickling code.
b1d1fe7fb514 Document the demise of all pretenses of safety, and the difference Guido van Rossum <guido@python.org> parents: 27507 diff changeset	138 References to these variables in the descriptions below are to be seen
b1d1fe7fb514 Document the demise of all pretenses of safety, and the difference Guido van Rossum <guido@python.org> parents: 27507 diff changeset	139 as describing unpickling in Python 2.2 and before.
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	140 """
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	141
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	142 # Meta-rule: Descriptions are stored in instances of descriptor objects,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	143 # with plain constructors. No meta-language is defined from which
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	144 # descriptors could be constructed. If you want, e.g., XML, write a little
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	145 # program to generate XML from the objects.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	146
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	147 ##############################################################################
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	148 # Some pickle opcodes have an argument, following the opcode in the
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	149 # bytestream. An argument is of a specific type, described by an instance
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	150 # of ArgumentDescriptor. These are not to be confused with arguments taken
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	151 # off the stack -- ArgumentDescriptor applies only to arguments embedded in
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	152 # the opcode stream, immediately following an opcode.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	153
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	154 # Represents the number of bytes consumed by an argument delimited by the
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	155 # next newline character.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	156 UP_TO_NEWLINE = -1
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	157
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	158 # Represents the number of bytes consumed by a two-argument opcode where
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	159 # the first argument gives the number of bytes in the second argument.
27435 430ed1c8e071 Removed the new LONG2 opcode: it's extravagant. If LONG1 isn't enough, Tim Peters <tim.peters@gmail.com> parents: 27434 diff changeset	160 TAKEN_FROM_ARGUMENT1 = -2 # num bytes is 1-byte unsigned int
430ed1c8e071 Removed the new LONG2 opcode: it's extravagant. If LONG1 isn't enough, Tim Peters <tim.peters@gmail.com> parents: 27434 diff changeset	161 TAKEN_FROM_ARGUMENT4 = -3 # num bytes is 4-byte signed little-endian int
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	162
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	163 class ArgumentDescriptor(object):
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	164 __slots__ = (
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	165 # name of descriptor record, also a module global name; a string
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	166 'name',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	167
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	168 # length of argument, in bytes; an int; UP_TO_NEWLINE and
27435 430ed1c8e071 Removed the new LONG2 opcode: it's extravagant. If LONG1 isn't enough, Tim Peters <tim.peters@gmail.com> parents: 27434 diff changeset	169 # TAKEN_FROM_ARGUMENT{1,4} are negative values for variable-length
430ed1c8e071 Removed the new LONG2 opcode: it's extravagant. If LONG1 isn't enough, Tim Peters <tim.peters@gmail.com> parents: 27434 diff changeset	170 # cases
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	171 'n',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	172
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	173 # a function taking a file-like object, reading this kind of argument
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	174 # from the object at the current position, advancing the current
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	175 # position by n bytes, and returning the value of the argument
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	176 'reader',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	177
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	178 # human-readable docs for this arg descriptor; a string
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	179 'doc',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	180 )
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	181
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	182 def __init__(self, name, n, reader, doc):
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	183 assert isinstance(name, str)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	184 self.name = name
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	185
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	186 assert isinstance(n, int) and (n >= 0 or
27435 430ed1c8e071 Removed the new LONG2 opcode: it's extravagant. If LONG1 isn't enough, Tim Peters <tim.peters@gmail.com> parents: 27434 diff changeset	187 n in (UP_TO_NEWLINE,
430ed1c8e071 Removed the new LONG2 opcode: it's extravagant. If LONG1 isn't enough, Tim Peters <tim.peters@gmail.com> parents: 27434 diff changeset	188 TAKEN_FROM_ARGUMENT1,
430ed1c8e071 Removed the new LONG2 opcode: it's extravagant. If LONG1 isn't enough, Tim Peters <tim.peters@gmail.com> parents: 27434 diff changeset	189 TAKEN_FROM_ARGUMENT4))
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	190 self.n = n
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	191
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	192 self.reader = reader
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	193
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	194 assert isinstance(doc, str)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	195 self.doc = doc
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	196
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	197 from struct import unpack as _unpack
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	198
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	199 def read_uint1(f):
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	200 r"""
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	201 >>> import StringIO
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	202 >>> read_uint1(StringIO.StringIO('\xff'))
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	203 255
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	204 """
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	205
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	206 data = f.read(1)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	207 if data:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	208 return ord(data)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	209 raise ValueError("not enough data in stream to read uint1")
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	210
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	211 uint1 = ArgumentDescriptor(
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	212 name='uint1',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	213 n=1,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	214 reader=read_uint1,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	215 doc="One-byte unsigned integer.")
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	216
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	217
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	218 def read_uint2(f):
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	219 r"""
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	220 >>> import StringIO
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	221 >>> read_uint2(StringIO.StringIO('\xff\x00'))
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	222 255
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	223 >>> read_uint2(StringIO.StringIO('\xff\xff'))
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	224 65535
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	225 """
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	226
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	227 data = f.read(2)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	228 if len(data) == 2:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	229 return _unpack("<H", data)[0]
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	230 raise ValueError("not enough data in stream to read uint2")
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	231
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	232 uint2 = ArgumentDescriptor(
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	233 name='uint2',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	234 n=2,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	235 reader=read_uint2,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	236 doc="Two-byte unsigned integer, little-endian.")
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	237
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	238
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	239 def read_int4(f):
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	240 r"""
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	241 >>> import StringIO
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	242 >>> read_int4(StringIO.StringIO('\xff\x00\x00\x00'))
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	243 255
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	244 >>> read_int4(StringIO.StringIO('\x00\x00\x00\x80')) == -(2**31)
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	245 True
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	246 """
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	247
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	248 data = f.read(4)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	249 if len(data) == 4:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	250 return _unpack("<i", data)[0]
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	251 raise ValueError("not enough data in stream to read int4")
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	252
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	253 int4 = ArgumentDescriptor(
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	254 name='int4',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	255 n=4,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	256 reader=read_int4,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	257 doc="Four-byte signed integer, little-endian, 2's complement.")
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	258
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	259
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	260 def read_stringnl(f, decode=True, stripquotes=True):
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	261 r"""
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	262 >>> import StringIO
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	263 >>> read_stringnl(StringIO.StringIO("'abcd'\nefg\n"))
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	264 'abcd'
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	265
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	266 >>> read_stringnl(StringIO.StringIO("\n"))
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	267 Traceback (most recent call last):
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	268 ...
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	269 ValueError: no string quotes around ''
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	270
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	271 >>> read_stringnl(StringIO.StringIO("\n"), stripquotes=False)
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	272 ''
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	273
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	274 >>> read_stringnl(StringIO.StringIO("''\n"))
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	275 ''
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	276
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	277 >>> read_stringnl(StringIO.StringIO('"abcd"'))
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	278 Traceback (most recent call last):
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	279 ...
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	280 ValueError: no newline found when trying to read stringnl
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	281
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	282 Embedded escapes are undone in the result.
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	283 >>> read_stringnl(StringIO.StringIO(r"'a\n\\b\x00c\td'" + "\n'e'"))
154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	284 'a\n\\b\x00c\td'
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	285 """
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	286
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	287 data = f.readline()
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	288 if not data.endswith('\n'):
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	289 raise ValueError("no newline found when trying to read stringnl")
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	290 data = data[:-1] # lose the newline
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	291
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	292 if stripquotes:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	293 for q in "'\"":
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	294 if data.startswith(q):
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	295 if not data.endswith(q):
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	296 raise ValueError("strinq quote %r not found at both "
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	297 "ends of %r" % (q, data))
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	298 data = data[1:-1]
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	299 break
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	300 else:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	301 raise ValueError("no string quotes around %r" % data)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	302
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	303 # I'm not sure when 'string_escape' was added to the std codecs; it's
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	304 # crazy not to use it if it's there.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	305 if decode:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	306 data = data.decode('string_escape')
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	307 return data
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	308
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	309 stringnl = ArgumentDescriptor(
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	310 name='stringnl',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	311 n=UP_TO_NEWLINE,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	312 reader=read_stringnl,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	313 doc="""A newline-terminated string.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	314
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	315 This is a repr-style string, with embedded escapes, and
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	316 bracketing quotes.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	317 """)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	318
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	319 def read_stringnl_noescape(f):
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	320 return read_stringnl(f, decode=False, stripquotes=False)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	321
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	322 stringnl_noescape = ArgumentDescriptor(
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	323 name='stringnl_noescape',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	324 n=UP_TO_NEWLINE,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	325 reader=read_stringnl_noescape,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	326 doc="""A newline-terminated string.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	327
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	328 This is a str-style string, without embedded escapes,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	329 or bracketing quotes. It should consist solely of
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	330 printable ASCII characters.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	331 """)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	332
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	333 def read_stringnl_noescape_pair(f):
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	334 r"""
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	335 >>> import StringIO
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	336 >>> read_stringnl_noescape_pair(StringIO.StringIO("Queue\nEmpty\njunk"))
27422 24c8fbec761f stringnl_noescape_pair(): changed the reader to separate the two Tim Peters <tim.peters@gmail.com> parents: 27421 diff changeset	337 'Queue Empty'
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	338 """
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	339
27422 24c8fbec761f stringnl_noescape_pair(): changed the reader to separate the two Tim Peters <tim.peters@gmail.com> parents: 27421 diff changeset	340 return "%s %s" % (read_stringnl_noescape(f), read_stringnl_noescape(f))
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	341
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	342 stringnl_noescape_pair = ArgumentDescriptor(
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	343 name='stringnl_noescape_pair',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	344 n=UP_TO_NEWLINE,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	345 reader=read_stringnl_noescape_pair,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	346 doc="""A pair of newline-terminated strings.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	347
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	348 These are str-style strings, without embedded
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	349 escapes, or bracketing quotes. They should
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	350 consist solely of printable ASCII characters.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	351 The pair is returned as a single string, with
27422 24c8fbec761f stringnl_noescape_pair(): changed the reader to separate the two Tim Peters <tim.peters@gmail.com> parents: 27421 diff changeset	352 a single blank separating the two strings.
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	353 """)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	354
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	355 def read_string4(f):
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	356 r"""
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	357 >>> import StringIO
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	358 >>> read_string4(StringIO.StringIO("\x00\x00\x00\x00abc"))
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	359 ''
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	360 >>> read_string4(StringIO.StringIO("\x03\x00\x00\x00abcdef"))
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	361 'abc'
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	362 >>> read_string4(StringIO.StringIO("\x00\x00\x00\x03abcdef"))
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	363 Traceback (most recent call last):
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	364 ...
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	365 ValueError: expected 50331648 bytes in a string4, but only 6 remain
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	366 """
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	367
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	368 n = read_int4(f)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	369 if n < 0:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	370 raise ValueError("string4 byte count < 0: %d" % n)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	371 data = f.read(n)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	372 if len(data) == n:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	373 return data
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	374 raise ValueError("expected %d bytes in a string4, but only %d remain" %
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	375 (n, len(data)))
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	376
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	377 string4 = ArgumentDescriptor(
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	378 name="string4",
27435 430ed1c8e071 Removed the new LONG2 opcode: it's extravagant. If LONG1 isn't enough, Tim Peters <tim.peters@gmail.com> parents: 27434 diff changeset	379 n=TAKEN_FROM_ARGUMENT4,
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	380 reader=read_string4,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	381 doc="""A counted string.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	382
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	383 The first argument is a 4-byte little-endian signed int giving
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	384 the number of bytes in the string, and the second argument is
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	385 that many bytes.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	386 """)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	387
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	388
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	389 def read_string1(f):
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	390 r"""
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	391 >>> import StringIO
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	392 >>> read_string1(StringIO.StringIO("\x00"))
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	393 ''
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	394 >>> read_string1(StringIO.StringIO("\x03abcdef"))
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	395 'abc'
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	396 """
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	397
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	398 n = read_uint1(f)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	399 assert n >= 0
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	400 data = f.read(n)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	401 if len(data) == n:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	402 return data
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	403 raise ValueError("expected %d bytes in a string1, but only %d remain" %
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	404 (n, len(data)))
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	405
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	406 string1 = ArgumentDescriptor(
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	407 name="string1",
27435 430ed1c8e071 Removed the new LONG2 opcode: it's extravagant. If LONG1 isn't enough, Tim Peters <tim.peters@gmail.com> parents: 27434 diff changeset	408 n=TAKEN_FROM_ARGUMENT1,
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	409 reader=read_string1,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	410 doc="""A counted string.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	411
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	412 The first argument is a 1-byte unsigned int giving the number
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	413 of bytes in the string, and the second argument is that many
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	414 bytes.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	415 """)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	416
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	417
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	418 def read_unicodestringnl(f):
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	419 r"""
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	420 >>> import StringIO
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	421 >>> read_unicodestringnl(StringIO.StringIO("abc\uabcd\njunk"))
154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	422 u'abc\uabcd'
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	423 """
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	424
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	425 data = f.readline()
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	426 if not data.endswith('\n'):
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	427 raise ValueError("no newline found when trying to read "
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	428 "unicodestringnl")
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	429 data = data[:-1] # lose the newline
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	430 return unicode(data, 'raw-unicode-escape')
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	431
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	432 unicodestringnl = ArgumentDescriptor(
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	433 name='unicodestringnl',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	434 n=UP_TO_NEWLINE,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	435 reader=read_unicodestringnl,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	436 doc="""A newline-terminated Unicode string.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	437
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	438 This is raw-unicode-escape encoded, so consists of
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	439 printable ASCII characters, and may contain embedded
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	440 escape sequences.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	441 """)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	442
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	443 def read_unicodestring4(f):
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	444 r"""
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	445 >>> import StringIO
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	446 >>> s = u'abcd\uabcd'
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	447 >>> enc = s.encode('utf-8')
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	448 >>> enc
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	449 'abcd\xea\xaf\x8d'
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	450 >>> n = chr(len(enc)) + chr(0) * 3 # little-endian 4-byte length
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	451 >>> t = read_unicodestring4(StringIO.StringIO(n + enc + 'junk'))
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	452 >>> s == t
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	453 True
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	454
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	455 >>> read_unicodestring4(StringIO.StringIO(n + enc[:-1]))
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	456 Traceback (most recent call last):
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	457 ...
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	458 ValueError: expected 7 bytes in a unicodestring4, but only 6 remain
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	459 """
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	460
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	461 n = read_int4(f)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	462 if n < 0:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	463 raise ValueError("unicodestring4 byte count < 0: %d" % n)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	464 data = f.read(n)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	465 if len(data) == n:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	466 return unicode(data, 'utf-8')
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	467 raise ValueError("expected %d bytes in a unicodestring4, but only %d "
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	468 "remain" % (n, len(data)))
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	469
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	470 unicodestring4 = ArgumentDescriptor(
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	471 name="unicodestring4",
27435 430ed1c8e071 Removed the new LONG2 opcode: it's extravagant. If LONG1 isn't enough, Tim Peters <tim.peters@gmail.com> parents: 27434 diff changeset	472 n=TAKEN_FROM_ARGUMENT4,
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	473 reader=read_unicodestring4,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	474 doc="""A counted Unicode string.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	475
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	476 The first argument is a 4-byte little-endian signed int
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	477 giving the number of bytes in the string, and the second
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	478 argument-- the UTF-8 encoding of the Unicode string --
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	479 contains that many bytes.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	480 """)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	481
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	482
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	483 def read_decimalnl_short(f):
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	484 r"""
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	485 >>> import StringIO
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	486 >>> read_decimalnl_short(StringIO.StringIO("1234\n56"))
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	487 1234
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	488
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	489 >>> read_decimalnl_short(StringIO.StringIO("1234L\n56"))
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	490 Traceback (most recent call last):
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	491 ...
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	492 ValueError: trailing 'L' not allowed in '1234L'
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	493 """
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	494
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	495 s = read_stringnl(f, decode=False, stripquotes=False)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	496 if s.endswith("L"):
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	497 raise ValueError("trailing 'L' not allowed in %r" % s)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	498
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	499 # It's not necessarily true that the result fits in a Python short int:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	500 # the pickle may have been written on a 64-bit box. There's also a hack
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	501 # for True and False here.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	502 if s == "00":
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	503 return False
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	504 elif s == "01":
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	505 return True
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	506
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	507 try:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	508 return int(s)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	509 except OverflowError:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	510 return long(s)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	511
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	512 def read_decimalnl_long(f):
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	513 r"""
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	514 >>> import StringIO
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	515
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	516 >>> read_decimalnl_long(StringIO.StringIO("1234\n56"))
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	517 Traceback (most recent call last):
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	518 ...
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	519 ValueError: trailing 'L' required in '1234'
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	520
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	521 Someday the trailing 'L' will probably go away from this output.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	522
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	523 >>> read_decimalnl_long(StringIO.StringIO("1234L\n56"))
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	524 1234L
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	525
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	526 >>> read_decimalnl_long(StringIO.StringIO("123456789012345678901234L\n6"))
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	527 123456789012345678901234L
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	528 """
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	529
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	530 s = read_stringnl(f, decode=False, stripquotes=False)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	531 if not s.endswith("L"):
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	532 raise ValueError("trailing 'L' required in %r" % s)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	533 return long(s)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	534
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	535
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	536 decimalnl_short = ArgumentDescriptor(
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	537 name='decimalnl_short',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	538 n=UP_TO_NEWLINE,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	539 reader=read_decimalnl_short,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	540 doc="""A newline-terminated decimal integer literal.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	541
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	542 This never has a trailing 'L', and the integer fit
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	543 in a short Python int on the box where the pickle
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	544 was written -- but there's no guarantee it will fit
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	545 in a short Python int on the box where the pickle
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	546 is read.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	547 """)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	548
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	549 decimalnl_long = ArgumentDescriptor(
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	550 name='decimalnl_long',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	551 n=UP_TO_NEWLINE,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	552 reader=read_decimalnl_long,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	553 doc="""A newline-terminated decimal integer literal.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	554
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	555 This has a trailing 'L', and can represent integers
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	556 of any size.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	557 """)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	558
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	559
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	560 def read_floatnl(f):
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	561 r"""
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	562 >>> import StringIO
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	563 >>> read_floatnl(StringIO.StringIO("-1.25\n6"))
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	564 -1.25
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	565 """
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	566 s = read_stringnl(f, decode=False, stripquotes=False)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	567 return float(s)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	568
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	569 floatnl = ArgumentDescriptor(
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	570 name='floatnl',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	571 n=UP_TO_NEWLINE,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	572 reader=read_floatnl,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	573 doc="""A newline-terminated decimal floating literal.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	574
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	575 In general this requires 17 significant digits for roundtrip
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	576 identity, and pickling then unpickling infinities, NaNs, and
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	577 minus zero doesn't work across boxes, or on some boxes even
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	578 on itself (e.g., Windows can't read the strings it produces
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	579 for infinities or NaNs).
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	580 """)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	581
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	582 def read_float8(f):
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	583 r"""
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	584 >>> import StringIO, struct
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	585 >>> raw = struct.pack(">d", -1.25)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	586 >>> raw
27473 154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	587 '\xbf\xf4\x00\x00\x00\x00\x00\x00'
154bac0b26a5 Use raw-mode docstring whenever there's an escape code in an example -- Tim Peters <tim.peters@gmail.com> parents: 27472 diff changeset	588 >>> read_float8(StringIO.StringIO(raw + "\n"))
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	589 -1.25
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	590 """
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	591
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	592 data = f.read(8)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	593 if len(data) == 8:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	594 return _unpack(">d", data)[0]
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	595 raise ValueError("not enough data in stream to read float8")
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	596
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	597
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	598 float8 = ArgumentDescriptor(
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	599 name='float8',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	600 n=8,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	601 reader=read_float8,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	602 doc="""An 8-byte binary representation of a float, big-endian.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	603
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	604 The format is unique to Python, and shared with the struct
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	605 module (format string '>d') "in theory" (the struct and cPickle
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	606 implementations don't share the code -- they should). It's
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	607 strongly related to the IEEE-754 double format, and, in normal
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	608 cases, is in fact identical to the big-endian 754 double format.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	609 On other boxes the dynamic range is limited to that of a 754
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	610 double, and "add a half and chop" rounding is used to reduce
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	611 the precision to 53 bits. However, even on a 754 box,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	612 infinities, NaNs, and minus zero may not be handled correctly
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	613 (may not survive roundtrip pickling intact).
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	614 """)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	615
27429 b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	616 # Protocol 2 formats
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	617
27506 b480a4f707f6 pickle: Comment repair. Tim Peters <tim.peters@gmail.com> parents: 27473 diff changeset	618 from pickle import decode_long
27429 b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	619
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	620 def read_long1(f):
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	621 r"""
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	622 >>> import StringIO
27552 50c734250e05 It's Official: for LONG1/LONG4, a "byte count" of 0 is taken as a Tim Peters <tim.peters@gmail.com> parents: 27535 diff changeset	623 >>> read_long1(StringIO.StringIO("\x00"))
50c734250e05 It's Official: for LONG1/LONG4, a "byte count" of 0 is taken as a Tim Peters <tim.peters@gmail.com> parents: 27535 diff changeset	624 0L
27429 b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	625 >>> read_long1(StringIO.StringIO("\x02\xff\x00"))
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	626 255L
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	627 >>> read_long1(StringIO.StringIO("\x02\xff\x7f"))
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	628 32767L
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	629 >>> read_long1(StringIO.StringIO("\x02\x00\xff"))
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	630 -256L
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	631 >>> read_long1(StringIO.StringIO("\x02\x00\x80"))
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	632 -32768L
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	633 """
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	634
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	635 n = read_uint1(f)
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	636 data = f.read(n)
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	637 if len(data) != n:
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	638 raise ValueError("not enough data in stream to read long1")
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	639 return decode_long(data)
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	640
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	641 long1 = ArgumentDescriptor(
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	642 name="long1",
27435 430ed1c8e071 Removed the new LONG2 opcode: it's extravagant. If LONG1 isn't enough, Tim Peters <tim.peters@gmail.com> parents: 27434 diff changeset	643 n=TAKEN_FROM_ARGUMENT1,
27429 b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	644 reader=read_long1,
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	645 doc="""A binary long, little-endian, using 1-byte size.
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	646
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	647 This first reads one byte as an unsigned size, then reads that
27434 7116cb5bac50 Documented the 2's-comp business for the new long opcodes. Tim Peters <tim.peters@gmail.com> parents: 27433 diff changeset	648 many bytes and interprets them as a little-endian 2's-complement long.
27552 50c734250e05 It's Official: for LONG1/LONG4, a "byte count" of 0 is taken as a Tim Peters <tim.peters@gmail.com> parents: 27535 diff changeset	649 If the size is 0, that's taken as a shortcut for the long 0L.
27429 b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	650 """)
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	651
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	652 def read_long4(f):
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	653 r"""
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	654 >>> import StringIO
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	655 >>> read_long4(StringIO.StringIO("\x02\x00\x00\x00\xff\x00"))
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	656 255L
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	657 >>> read_long4(StringIO.StringIO("\x02\x00\x00\x00\xff\x7f"))
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	658 32767L
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	659 >>> read_long4(StringIO.StringIO("\x02\x00\x00\x00\x00\xff"))
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	660 -256L
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	661 >>> read_long4(StringIO.StringIO("\x02\x00\x00\x00\x00\x80"))
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	662 -32768L
27552 50c734250e05 It's Official: for LONG1/LONG4, a "byte count" of 0 is taken as a Tim Peters <tim.peters@gmail.com> parents: 27535 diff changeset	663 >>> read_long1(StringIO.StringIO("\x00\x00\x00\x00"))
50c734250e05 It's Official: for LONG1/LONG4, a "byte count" of 0 is taken as a Tim Peters <tim.peters@gmail.com> parents: 27535 diff changeset	664 0L
27429 b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	665 """
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	666
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	667 n = read_int4(f)
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	668 if n < 0:
27436 60ff1ba428c3 Use proper function name in exception messages Neal Norwitz <nnorwitz@gmail.com> parents: 27435 diff changeset	669 raise ValueError("long4 byte count < 0: %d" % n)
27429 b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	670 data = f.read(n)
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	671 if len(data) != n:
27436 60ff1ba428c3 Use proper function name in exception messages Neal Norwitz <nnorwitz@gmail.com> parents: 27435 diff changeset	672 raise ValueError("not enough data in stream to read long4")
27429 b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	673 return decode_long(data)
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	674
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	675 long4 = ArgumentDescriptor(
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	676 name="long4",
27435 430ed1c8e071 Removed the new LONG2 opcode: it's extravagant. If LONG1 isn't enough, Tim Peters <tim.peters@gmail.com> parents: 27434 diff changeset	677 n=TAKEN_FROM_ARGUMENT4,
27429 b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	678 reader=read_long4,
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	679 doc="""A binary representation of a long, little-endian.
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	680
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	681 This first reads four bytes as a signed size (but requires the
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	682 size to be >= 0), then reads that many bytes and interprets them
27552 50c734250e05 It's Official: for LONG1/LONG4, a "byte count" of 0 is taken as a Tim Peters <tim.peters@gmail.com> parents: 27535 diff changeset	683 as a little-endian 2's-complement long. If the size is 0, that's taken
50c734250e05 It's Official: for LONG1/LONG4, a "byte count" of 0 is taken as a Tim Peters <tim.peters@gmail.com> parents: 27535 diff changeset	684 as a shortcut for the long 0L, although LONG1 should really be used
50c734250e05 It's Official: for LONG1/LONG4, a "byte count" of 0 is taken as a Tim Peters <tim.peters@gmail.com> parents: 27535 diff changeset	685 then instead (and in any case where # of bytes < 256).
27429 b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	686 """)
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	687
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	688
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	689 ##############################################################################
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	690 # Object descriptors. The stack used by the pickle machine holds objects,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	691 # and in the stack_before and stack_after attributes of OpcodeInfo
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	692 # descriptors we need names to describe the various types of objects that can
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	693 # appear on the stack.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	694
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	695 class StackObject(object):
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	696 __slots__ = (
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	697 # name of descriptor record, for info only
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	698 'name',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	699
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	700 # type of object, or tuple of type objects (meaning the object can
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	701 # be of any type in the tuple)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	702 'obtype',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	703
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	704 # human-readable docs for this kind of stack object; a string
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	705 'doc',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	706 )
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	707
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	708 def __init__(self, name, obtype, doc):
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	709 assert isinstance(name, str)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	710 self.name = name
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	711
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	712 assert isinstance(obtype, type) or isinstance(obtype, tuple)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	713 if isinstance(obtype, tuple):
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	714 for contained in obtype:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	715 assert isinstance(contained, type)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	716 self.obtype = obtype
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	717
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	718 assert isinstance(doc, str)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	719 self.doc = doc
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	720
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	721 def __repr__(self):
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	722 return self.name
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	723
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	724
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	725 pyint = StackObject(
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	726 name='int',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	727 obtype=int,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	728 doc="A short (as opposed to long) Python integer object.")
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	729
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	730 pylong = StackObject(
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	731 name='long',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	732 obtype=long,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	733 doc="A long (as opposed to short) Python integer object.")
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	734
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	735 pyinteger_or_bool = StackObject(
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	736 name='int_or_bool',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	737 obtype=(int, long, bool),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	738 doc="A Python integer object (short or long), or "
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	739 "a Python bool.")
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	740
27429 b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	741 pybool = StackObject(
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	742 name='bool',
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	743 obtype=(bool,),
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	744 doc="A Python bool object.")
b465c9d2196a Begin documenting protocol 2. Guido van Rossum <guido@python.org> parents: 27424 diff changeset	745
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	746 pyfloat = StackObject(
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	747 name='float',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	748 obtype=float,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	749 doc="A Python float object.")
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	750
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	751 pystring = StackObject(
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	752 name='str',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	753 obtype=str,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	754 doc="A Python string object.")
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	755
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	756 pyunicode = StackObject(
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	757 name='unicode',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	758 obtype=unicode,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	759 doc="A Python Unicode string object.")
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	760
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	761 pynone = StackObject(
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	762 name="None",
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	763 obtype=type(None),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	764 doc="The Python None object.")
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	765
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	766 pytuple = StackObject(
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	767 name="tuple",
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	768 obtype=tuple,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	769 doc="A Python tuple object.")
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	770
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	771 pylist = StackObject(
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	772 name="list",
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	773 obtype=list,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	774 doc="A Python list object.")
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	775
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	776 pydict = StackObject(
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	777 name="dict",
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	778 obtype=dict,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	779 doc="A Python dict object.")
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	780
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	781 anyobject = StackObject(
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	782 name='any',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	783 obtype=object,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	784 doc="Any kind of object whatsoever.")
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	785
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	786 markobject = StackObject(
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	787 name="mark",
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	788 obtype=StackObject,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	789 doc="""'The mark' is a unique object.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	790
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	791 Opcodes that operate on a variable number of objects
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	792 generally don't embed the count of objects in the opcode,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	793 or pull it off the stack. Instead the MARK opcode is used
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	794 to push a special marker object on the stack, and then
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	795 some other opcodes grab all the objects from the top of
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	796 the stack down to (but not including) the topmost marker
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	797 object.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	798 """)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	799
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	800 stackslice = StackObject(
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	801 name="stackslice",
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	802 obtype=StackObject,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	803 doc="""An object representing a contiguous slice of the stack.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	804
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	805 This is used in conjuction with markobject, to represent all
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	806 of the stack following the topmost markobject. For example,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	807 the POP_MARK opcode changes the stack from
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	808
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	809 [..., markobject, stackslice]
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	810 to
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	811 [...]
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	812
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	813 No matter how many object are on the stack after the topmost
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	814 markobject, POP_MARK gets rid of all of them (including the
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	815 topmost markobject too).
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	816 """)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	817
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	818 ##############################################################################
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	819 # Descriptors for pickle opcodes.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	820
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	821 class OpcodeInfo(object):
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	822
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	823 __slots__ = (
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	824 # symbolic name of opcode; a string
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	825 'name',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	826
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	827 # the code used in a bytestream to represent the opcode; a
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	828 # one-character string
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	829 'code',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	830
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	831 # If the opcode has an argument embedded in the byte string, an
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	832 # instance of ArgumentDescriptor specifying its type. Note that
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	833 # arg.reader(s) can be used to read and decode the argument from
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	834 # the bytestream s, and arg.doc documents the format of the raw
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	835 # argument bytes. If the opcode doesn't have an argument embedded
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	836 # in the bytestream, arg should be None.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	837 'arg',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	838
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	839 # what the stack looks like before this opcode runs; a list
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	840 'stack_before',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	841
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	842 # what the stack looks like after this opcode runs; a list
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	843 'stack_after',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	844
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	845 # the protocol number in which this opcode was introduced; an int
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	846 'proto',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	847
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	848 # human-readable docs for this opcode; a string
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	849 'doc',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	850 )
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	851
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	852 def __init__(self, name, code, arg,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	853 stack_before, stack_after, proto, doc):
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	854 assert isinstance(name, str)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	855 self.name = name
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	856
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	857 assert isinstance(code, str)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	858 assert len(code) == 1
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	859 self.code = code
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	860
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	861 assert arg is None or isinstance(arg, ArgumentDescriptor)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	862 self.arg = arg
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	863
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	864 assert isinstance(stack_before, list)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	865 for x in stack_before:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	866 assert isinstance(x, StackObject)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	867 self.stack_before = stack_before
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	868
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	869 assert isinstance(stack_after, list)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	870 for x in stack_after:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	871 assert isinstance(x, StackObject)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	872 self.stack_after = stack_after
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	873
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	874 assert isinstance(proto, int) and 0 <= proto <= 2
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	875 self.proto = proto
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	876
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	877 assert isinstance(doc, str)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	878 self.doc = doc
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	879
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	880 I = OpcodeInfo
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	881 opcodes = [
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	882
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	883 # Ways to spell integers.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	884
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	885 I(name='INT',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	886 code='I',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	887 arg=decimalnl_short,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	888 stack_before=[],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	889 stack_after=[pyinteger_or_bool],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	890 proto=0,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	891 doc="""Push an integer or bool.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	892
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	893 The argument is a newline-terminated decimal literal string.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	894
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	895 The intent may have been that this always fit in a short Python int,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	896 but INT can be generated in pickles written on a 64-bit box that
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	897 require a Python long on a 32-bit box. The difference between this
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	898 and LONG then is that INT skips a trailing 'L', and produces a short
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	899 int whenever possible.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	900
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	901 Another difference is due to that, when bool was introduced as a
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	902 distinct type in 2.3, builtin names True and False were also added to
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	903 2.2.2, mapping to ints 1 and 0. For compatibility in both directions,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	904 True gets pickled as INT + "I01\\n", and False as INT + "I00\\n".
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	905 Leading zeroes are never produced for a genuine integer. The 2.3
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	906 (and later) unpicklers special-case these and return bool instead;
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	907 earlier unpicklers ignore the leading "0" and return the int.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	908 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	909
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	910 I(name='BININT',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	911 code='J',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	912 arg=int4,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	913 stack_before=[],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	914 stack_after=[pyint],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	915 proto=1,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	916 doc="""Push a four-byte signed integer.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	917
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	918 This handles the full range of Python (short) integers on a 32-bit
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	919 box, directly as binary bytes (1 for the opcode and 4 for the integer).
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	920 If the integer is non-negative and fits in 1 or 2 bytes, pickling via
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	921 BININT1 or BININT2 saves space.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	922 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	923
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	924 I(name='BININT1',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	925 code='K',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	926 arg=uint1,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	927 stack_before=[],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	928 stack_after=[pyint],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	929 proto=1,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	930 doc="""Push a one-byte unsigned integer.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	931
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	932 This is a space optimization for pickling very small non-negative ints,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	933 in range(256).
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	934 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	935
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	936 I(name='BININT2',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	937 code='M',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	938 arg=uint2,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	939 stack_before=[],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	940 stack_after=[pyint],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	941 proto=1,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	942 doc="""Push a two-byte unsigned integer.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	943
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	944 This is a space optimization for pickling small positive ints, in
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	945 range(256, 2**16). Integers in range(256) can also be pickled via
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	946 BININT2, but BININT1 instead saves a byte.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	947 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	948
27462 26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	949 I(name='LONG',
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	950 code='L',
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	951 arg=decimalnl_long,
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	952 stack_before=[],
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	953 stack_after=[pylong],
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	954 proto=0,
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	955 doc="""Push a long integer.
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	956
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	957 The same as INT, except that the literal ends with 'L', and always
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	958 unpickles to a Python long. There doesn't seem a real purpose to the
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	959 trailing 'L'.
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	960
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	961 Note that LONG takes time quadratic in the number of digits when
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	962 unpickling (this is simply due to the nature of decimal->binary
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	963 conversion). Proto 2 added linear-time (in C; still quadratic-time
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	964 in Python) LONG1 and LONG4 opcodes.
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	965 """),
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	966
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	967 I(name="LONG1",
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	968 code='\x8a',
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	969 arg=long1,
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	970 stack_before=[],
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	971 stack_after=[pylong],
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	972 proto=2,
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	973 doc="""Long integer using one-byte length.
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	974
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	975 A more efficient encoding of a Python long; the long1 encoding
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	976 says it all."""),
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	977
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	978 I(name="LONG4",
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	979 code='\x8b',
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	980 arg=long4,
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	981 stack_before=[],
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	982 stack_after=[pylong],
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	983 proto=2,
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	984 doc="""Long integer using found-byte length.
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	985
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	986 A more efficient encoding of a Python long; the long4 encoding
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	987 says it all."""),
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	988
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	989 # Ways to spell strings (8-bit, not Unicode).
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	990
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	991 I(name='STRING',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	992 code='S',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	993 arg=stringnl,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	994 stack_before=[],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	995 stack_after=[pystring],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	996 proto=0,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	997 doc="""Push a Python string object.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	998
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	999 The argument is a repr-style string, with bracketing quote characters,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1000 and perhaps embedded escapes. The argument extends until the next
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1001 newline character.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1002 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1003
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1004 I(name='BINSTRING',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1005 code='T',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1006 arg=string4,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1007 stack_before=[],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1008 stack_after=[pystring],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1009 proto=1,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1010 doc="""Push a Python string object.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1011
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1012 There are two arguments: the first is a 4-byte little-endian signed int
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1013 giving the number of bytes in the string, and the second is that many
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1014 bytes, which are taken literally as the string content.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1015 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1016
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1017 I(name='SHORT_BINSTRING',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1018 code='U',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1019 arg=string1,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1020 stack_before=[],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1021 stack_after=[pystring],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1022 proto=1,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1023 doc="""Push a Python string object.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1024
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1025 There are two arguments: the first is a 1-byte unsigned int giving
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1026 the number of bytes in the string, and the second is that many bytes,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1027 which are taken literally as the string content.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1028 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1029
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1030 # Ways to spell None.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1031
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1032 I(name='NONE',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1033 code='N',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1034 arg=None,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1035 stack_before=[],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1036 stack_after=[pynone],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1037 proto=0,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1038 doc="Push None on the stack."),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1039
27462 26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1040 # Ways to spell bools, starting with proto 2. See INT for how this was
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1041 # done before proto 2.
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1042
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1043 I(name='NEWTRUE',
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1044 code='\x88',
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1045 arg=None,
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1046 stack_before=[],
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1047 stack_after=[pybool],
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1048 proto=2,
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1049 doc="""True.
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1050
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1051 Push True onto the stack."""),
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1052
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1053 I(name='NEWFALSE',
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1054 code='\x89',
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1055 arg=None,
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1056 stack_before=[],
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1057 stack_after=[pybool],
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1058 proto=2,
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1059 doc="""True.
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1060
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1061 Push False onto the stack."""),
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1062
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1063 # Ways to spell Unicode strings.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1064
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1065 I(name='UNICODE',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1066 code='V',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1067 arg=unicodestringnl,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1068 stack_before=[],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1069 stack_after=[pyunicode],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1070 proto=0, # this may be pure-text, but it's a later addition
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1071 doc="""Push a Python Unicode string object.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1072
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1073 The argument is a raw-unicode-escape encoding of a Unicode string,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1074 and so may contain embedded escape sequences. The argument extends
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1075 until the next newline character.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1076 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1077
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1078 I(name='BINUNICODE',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1079 code='X',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1080 arg=unicodestring4,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1081 stack_before=[],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1082 stack_after=[pyunicode],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1083 proto=1,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1084 doc="""Push a Python Unicode string object.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1085
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1086 There are two arguments: the first is a 4-byte little-endian signed int
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1087 giving the number of bytes in the string. The second is that many
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1088 bytes, and is the UTF-8 encoding of the Unicode string.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1089 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1090
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1091 # Ways to spell floats.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1092
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1093 I(name='FLOAT',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1094 code='F',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1095 arg=floatnl,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1096 stack_before=[],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1097 stack_after=[pyfloat],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1098 proto=0,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1099 doc="""Newline-terminated decimal float literal.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1100
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1101 The argument is repr(a_float), and in general requires 17 significant
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1102 digits for roundtrip conversion to be an identity (this is so for
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1103 IEEE-754 double precision values, which is what Python float maps to
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1104 on most boxes).
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1105
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1106 In general, FLOAT cannot be used to transport infinities, NaNs, or
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1107 minus zero across boxes (or even on a single box, if the platform C
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1108 library can't read the strings it produces for such things -- Windows
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1109 is like that), but may do less damage than BINFLOAT on boxes with
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1110 greater precision or dynamic range than IEEE-754 double.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1111 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1112
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1113 I(name='BINFLOAT',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1114 code='G',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1115 arg=float8,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1116 stack_before=[],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1117 stack_after=[pyfloat],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1118 proto=1,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1119 doc="""Float stored in binary form, with 8 bytes of data.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1120
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1121 This generally requires less than half the space of FLOAT encoding.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1122 In general, BINFLOAT cannot be used to transport infinities, NaNs, or
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1123 minus zero, raises an exception if the exponent exceeds the range of
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1124 an IEEE-754 double, and retains no more than 53 bits of precision (if
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1125 there are more than that, "add a half and chop" rounding is used to
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1126 cut it back to 53 significant bits).
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1127 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1128
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1129 # Ways to build lists.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1130
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1131 I(name='EMPTY_LIST',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1132 code=']',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1133 arg=None,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1134 stack_before=[],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1135 stack_after=[pylist],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1136 proto=1,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1137 doc="Push an empty list."),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1138
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1139 I(name='APPEND',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1140 code='a',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1141 arg=None,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1142 stack_before=[pylist, anyobject],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1143 stack_after=[pylist],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1144 proto=0,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1145 doc="""Append an object to a list.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1146
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1147 Stack before: ... pylist anyobject
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1148 Stack after: ... pylist+[anyobject]
27463 2ee9742aadd8 Comments. Tim Peters <tim.peters@gmail.com> parents: 27462 diff changeset	1149
2ee9742aadd8 Comments. Tim Peters <tim.peters@gmail.com> parents: 27462 diff changeset	1150 although pylist is really extended in-place.
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1151 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1152
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1153 I(name='APPENDS',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1154 code='e',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1155 arg=None,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1156 stack_before=[pylist, markobject, stackslice],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1157 stack_after=[pylist],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1158 proto=1,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1159 doc="""Extend a list by a slice of stack objects.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1160
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1161 Stack before: ... pylist markobject stackslice
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1162 Stack after: ... pylist+stackslice
27463 2ee9742aadd8 Comments. Tim Peters <tim.peters@gmail.com> parents: 27462 diff changeset	1163
2ee9742aadd8 Comments. Tim Peters <tim.peters@gmail.com> parents: 27462 diff changeset	1164 although pylist is really extended in-place.
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1165 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1166
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1167 I(name='LIST',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1168 code='l',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1169 arg=None,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1170 stack_before=[markobject, stackslice],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1171 stack_after=[pylist],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1172 proto=0,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1173 doc="""Build a list out of the topmost stack slice, after markobject.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1174
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1175 All the stack entries following the topmost markobject are placed into
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1176 a single Python list, which single list object replaces all of the
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1177 stack from the topmost markobject onward. For example,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1178
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1179 Stack before: ... markobject 1 2 3 'abc'
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1180 Stack after: ... [1, 2, 3, 'abc']
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1181 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1182
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1183 # Ways to build tuples.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1184
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1185 I(name='EMPTY_TUPLE',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1186 code=')',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1187 arg=None,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1188 stack_before=[],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1189 stack_after=[pytuple],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1190 proto=1,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1191 doc="Push an empty tuple."),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1192
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1193 I(name='TUPLE',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1194 code='t',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1195 arg=None,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1196 stack_before=[markobject, stackslice],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1197 stack_after=[pytuple],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1198 proto=0,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1199 doc="""Build a tuple out of the topmost stack slice, after markobject.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1200
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1201 All the stack entries following the topmost markobject are placed into
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1202 a single Python tuple, which single tuple object replaces all of the
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1203 stack from the topmost markobject onward. For example,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1204
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1205 Stack before: ... markobject 1 2 3 'abc'
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1206 Stack after: ... (1, 2, 3, 'abc')
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1207 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1208
27462 26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1209 I(name='TUPLE1',
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1210 code='\x85',
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1211 arg=None,
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1212 stack_before=[anyobject],
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1213 stack_after=[pytuple],
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1214 proto=2,
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1215 doc="""One-tuple.
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1216
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1217 This code pops one value off the stack and pushes a tuple of
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1218 length 1 whose one item is that value back onto it. IOW:
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1219
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1220 stack[-1] = tuple(stack[-1:])
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1221 """),
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1222
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1223 I(name='TUPLE2',
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1224 code='\x86',
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1225 arg=None,
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1226 stack_before=[anyobject, anyobject],
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1227 stack_after=[pytuple],
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1228 proto=2,
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1229 doc="""One-tuple.
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1230
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1231 This code pops two values off the stack and pushes a tuple
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1232 of length 2 whose items are those values back onto it. IOW:
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1233
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1234 stack[-2:] = [tuple(stack[-2:])]
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1235 """),
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1236
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1237 I(name='TUPLE3',
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1238 code='\x87',
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1239 arg=None,
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1240 stack_before=[anyobject, anyobject, anyobject],
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1241 stack_after=[pytuple],
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1242 proto=2,
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1243 doc="""One-tuple.
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1244
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1245 This code pops three values off the stack and pushes a tuple
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1246 of length 3 whose items are those values back onto it. IOW:
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1247
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1248 stack[-3:] = [tuple(stack[-3:])]
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1249 """),
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1250
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1251 # Ways to build dicts.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1252
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1253 I(name='EMPTY_DICT',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1254 code='}',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1255 arg=None,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1256 stack_before=[],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1257 stack_after=[pydict],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1258 proto=1,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1259 doc="Push an empty dict."),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1260
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1261 I(name='DICT',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1262 code='d',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1263 arg=None,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1264 stack_before=[markobject, stackslice],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1265 stack_after=[pydict],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1266 proto=0,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1267 doc="""Build a dict out of the topmost stack slice, after markobject.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1268
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1269 All the stack entries following the topmost markobject are placed into
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1270 a single Python dict, which single dict object replaces all of the
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1271 stack from the topmost markobject onward. The stack slice alternates
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1272 key, value, key, value, .... For example,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1273
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1274 Stack before: ... markobject 1 2 3 'abc'
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1275 Stack after: ... {1: 2, 3: 'abc'}
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1276 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1277
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1278 I(name='SETITEM',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1279 code='s',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1280 arg=None,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1281 stack_before=[pydict, anyobject, anyobject],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1282 stack_after=[pydict],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1283 proto=0,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1284 doc="""Add a key+value pair to an existing dict.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1285
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1286 Stack before: ... pydict key value
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1287 Stack after: ... pydict
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1288
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1289 where pydict has been modified via pydict[key] = value.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1290 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1291
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1292 I(name='SETITEMS',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1293 code='u',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1294 arg=None,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1295 stack_before=[pydict, markobject, stackslice],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1296 stack_after=[pydict],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1297 proto=1,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1298 doc="""Add an arbitrary number of key+value pairs to an existing dict.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1299
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1300 The slice of the stack following the topmost markobject is taken as
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1301 an alternating sequence of keys and values, added to the dict
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1302 immediately under the topmost markobject. Everything at and after the
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1303 topmost markobject is popped, leaving the mutated dict at the top
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1304 of the stack.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1305
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1306 Stack before: ... pydict markobject key_1 value_1 ... key_n value_n
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1307 Stack after: ... pydict
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1308
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1309 where pydict has been modified via pydict[key_i] = value_i for i in
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1310 1, 2, ..., n, and in that order.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1311 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1312
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1313 # Stack manipulation.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1314
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1315 I(name='POP',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1316 code='0',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1317 arg=None,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1318 stack_before=[anyobject],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1319 stack_after=[],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1320 proto=0,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1321 doc="Discard the top stack item, shrinking the stack by one item."),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1322
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1323 I(name='DUP',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1324 code='2',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1325 arg=None,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1326 stack_before=[anyobject],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1327 stack_after=[anyobject, anyobject],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1328 proto=0,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1329 doc="Push the top stack item onto the stack again, duplicating it."),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1330
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1331 I(name='MARK',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1332 code='(',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1333 arg=None,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1334 stack_before=[],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1335 stack_after=[markobject],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1336 proto=0,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1337 doc="""Push markobject onto the stack.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1338
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1339 markobject is a unique object, used by other opcodes to identify a
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1340 region of the stack containing a variable number of objects for them
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1341 to work on. See markobject.doc for more detail.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1342 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1343
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1344 I(name='POP_MARK',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1345 code='1',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1346 arg=None,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1347 stack_before=[markobject, stackslice],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1348 stack_after=[],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1349 proto=0,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1350 doc="""Pop all the stack objects at and above the topmost markobject.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1351
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1352 When an opcode using a variable number of stack objects is done,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1353 POP_MARK is used to remove those objects, and to remove the markobject
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1354 that delimited their starting position on the stack.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1355 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1356
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1357 # Memo manipulation. There are really only two operations (get and put),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1358 # each in all-text, "short binary", and "long binary" flavors.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1359
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1360 I(name='GET',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1361 code='g',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1362 arg=decimalnl_short,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1363 stack_before=[],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1364 stack_after=[anyobject],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1365 proto=0,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1366 doc="""Read an object from the memo and push it on the stack.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1367
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1368 The index of the memo object to push is given by the newline-teriminated
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1369 decimal string following. BINGET and LONG_BINGET are space-optimized
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1370 versions.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1371 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1372
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1373 I(name='BINGET',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1374 code='h',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1375 arg=uint1,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1376 stack_before=[],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1377 stack_after=[anyobject],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1378 proto=1,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1379 doc="""Read an object from the memo and push it on the stack.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1380
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1381 The index of the memo object to push is given by the 1-byte unsigned
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1382 integer following.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1383 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1384
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1385 I(name='LONG_BINGET',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1386 code='j',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1387 arg=int4,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1388 stack_before=[],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1389 stack_after=[anyobject],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1390 proto=1,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1391 doc="""Read an object from the memo and push it on the stack.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1392
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1393 The index of the memo object to push is given by the 4-byte signed
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1394 little-endian integer following.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1395 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1396
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1397 I(name='PUT',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1398 code='p',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1399 arg=decimalnl_short,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1400 stack_before=[],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1401 stack_after=[],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1402 proto=0,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1403 doc="""Store the stack top into the memo. The stack is not popped.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1404
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1405 The index of the memo location to write into is given by the newline-
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1406 terminated decimal string following. BINPUT and LONG_BINPUT are
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1407 space-optimized versions.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1408 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1409
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1410 I(name='BINPUT',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1411 code='q',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1412 arg=uint1,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1413 stack_before=[],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1414 stack_after=[],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1415 proto=1,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1416 doc="""Store the stack top into the memo. The stack is not popped.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1417
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1418 The index of the memo location to write into is given by the 1-byte
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1419 unsigned integer following.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1420 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1421
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1422 I(name='LONG_BINPUT',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1423 code='r',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1424 arg=int4,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1425 stack_before=[],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1426 stack_after=[],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1427 proto=1,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1428 doc="""Store the stack top into the memo. The stack is not popped.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1429
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1430 The index of the memo location to write into is given by the 4-byte
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1431 signed little-endian integer following.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1432 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1433
27462 26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1434 # Access the extension registry (predefined objects). Akin to the GET
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1435 # family.
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1436
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1437 I(name='EXT1',
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1438 code='\x82',
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1439 arg=uint1,
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1440 stack_before=[],
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1441 stack_after=[anyobject],
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1442 proto=2,
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1443 doc="""Extension code.
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1444
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1445 This code and the similar EXT2 and EXT4 allow using a registry
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1446 of popular objects that are pickled by name, typically classes.
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1447 It is envisioned that through a global negotiation and
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1448 registration process, third parties can set up a mapping between
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1449 ints and object names.
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1450
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1451 In order to guarantee pickle interchangeability, the extension
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1452 code registry ought to be global, although a range of codes may
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1453 be reserved for private use.
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1454
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1455 EXT1 has a 1-byte integer argument. This is used to index into the
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1456 extension registry, and the object at that index is pushed on the stack.
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1457 """),
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1458
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1459 I(name='EXT2',
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1460 code='\x83',
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1461 arg=uint2,
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1462 stack_before=[],
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1463 stack_after=[anyobject],
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1464 proto=2,
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1465 doc="""Extension code.
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1466
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1467 See EXT1. EXT2 has a two-byte integer argument.
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1468 """),
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1469
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1470 I(name='EXT4',
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1471 code='\x84',
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1472 arg=int4,
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1473 stack_before=[],
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1474 stack_after=[anyobject],
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1475 proto=2,
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1476 doc="""Extension code.
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1477
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1478 See EXT1. EXT4 has a four-byte integer argument.
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1479 """),
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1480
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1481 # Push a class object, or module function, on the stack, via its module
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1482 # and name.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1483
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1484 I(name='GLOBAL',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1485 code='c',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1486 arg=stringnl_noescape_pair,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1487 stack_before=[],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1488 stack_after=[anyobject],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1489 proto=0,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1490 doc="""Push a global object (module.attr) on the stack.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1491
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1492 Two newline-terminated strings follow the GLOBAL opcode. The first is
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1493 taken as a module name, and the second as a class name. The class
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1494 object module.class is pushed on the stack. More accurately, the
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1495 object returned by self.find_class(module, class) is pushed on the
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1496 stack, so unpickling subclasses can override this form of lookup.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1497 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1498
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1499 # Ways to build objects of classes pickle doesn't know about directly
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1500 # (user-defined classes). I despair of documenting this accurately
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1501 # and comprehensibly -- you really have to read the pickle code to
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1502 # find all the special cases.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1503
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1504 I(name='REDUCE',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1505 code='R',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1506 arg=None,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1507 stack_before=[anyobject, anyobject],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1508 stack_after=[anyobject],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1509 proto=0,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1510 doc="""Push an object built from a callable and an argument tuple.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1511
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1512 The opcode is named to remind of the __reduce__() method.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1513
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1514 Stack before: ... callable pytuple
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1515 Stack after: ... callable(*pytuple)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1516
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1517 The callable and the argument tuple are the first two items returned
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1518 by a __reduce__ method. Applying the callable to the argtuple is
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1519 supposed to reproduce the original object, or at least get it started.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1520 If the __reduce__ method returns a 3-tuple, the last component is an
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1521 argument to be passed to the object's __setstate__, and then the REDUCE
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1522 opcode is followed by code to create setstate's argument, and then a
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1523 BUILD opcode to apply __setstate__ to that argument.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1524
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1525 There are lots of special cases here. The argtuple can be None, in
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1526 which case callable.__basicnew__() is called instead to produce the
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1527 object to be pushed on the stack. This appears to be a trick unique
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1528 to ExtensionClasses, and is deprecated regardless.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1529
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1530 If type(callable) is not ClassType, REDUCE complains unless the
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1531 callable has been registered with the copy_reg module's
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1532 safe_constructors dict, or the callable has a magic
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1533 '__safe_for_unpickling__' attribute with a true value. I'm not sure
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1534 why it does this, but I've sure seen this complaint often enough when
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1535 I didn't want to <wink>.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1536 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1537
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1538 I(name='BUILD',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1539 code='b',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1540 arg=None,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1541 stack_before=[anyobject, anyobject],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1542 stack_after=[anyobject],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1543 proto=0,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1544 doc="""Finish building an object, via __setstate__ or dict update.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1545
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1546 Stack before: ... anyobject argument
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1547 Stack after: ... anyobject
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1548
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1549 where anyobject may have been mutated, as follows:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1550
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1551 If the object has a __setstate__ method,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1552
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1553 anyobject.__setstate__(argument)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1554
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1555 is called.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1556
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1557 Else the argument must be a dict, the object must have a __dict__, and
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1558 the object is updated via
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1559
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1560 anyobject.__dict__.update(argument)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1561
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1562 This may raise RuntimeError in restricted execution mode (which
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1563 disallows access to __dict__ directly); in that case, the object
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1564 is updated instead via
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1565
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1566 for k, v in argument.items():
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1567 anyobject[k] = v
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1568 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1569
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1570 I(name='INST',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1571 code='i',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1572 arg=stringnl_noescape_pair,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1573 stack_before=[markobject, stackslice],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1574 stack_after=[anyobject],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1575 proto=0,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1576 doc="""Build a class instance.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1577
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1578 This is the protocol 0 version of protocol 1's OBJ opcode.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1579 INST is followed by two newline-terminated strings, giving a
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1580 module and class name, just as for the GLOBAL opcode (and see
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1581 GLOBAL for more details about that). self.find_class(module, name)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1582 is used to get a class object.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1583
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1584 In addition, all the objects on the stack following the topmost
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1585 markobject are gathered into a tuple and popped (along with the
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1586 topmost markobject), just as for the TUPLE opcode.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1587
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1588 Now it gets complicated. If all of these are true:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1589
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1590 + The argtuple is empty (markobject was at the top of the stack
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1591 at the start).
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1592
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1593 + It's an old-style class object (the type of the class object is
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1594 ClassType).
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1595
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1596 + The class object does not have a __getinitargs__ attribute.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1597
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1598 then we want to create an old-style class instance without invoking
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1599 its __init__() method (pickle has waffled on this over the years; not
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1600 calling __init__() is current wisdom). In this case, an instance of
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1601 an old-style dummy class is created, and then we try to rebind its
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1602 __class__ attribute to the desired class object. If this succeeds,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1603 the new instance object is pushed on the stack, and we're done. In
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1604 restricted execution mode it can fail (assignment to __class__ is
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1605 disallowed), and I'm not really sure what happens then -- it looks
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1606 like the code ends up calling the class object's __init__ anyway,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1607 via falling into the next case.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1608
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1609 Else (the argtuple is not empty, it's not an old-style class object,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1610 or the class object does have a __getinitargs__ attribute), the code
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1611 first insists that the class object have a __safe_for_unpickling__
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1612 attribute. Unlike as for the __safe_for_unpickling__ check in REDUCE,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1613 it doesn't matter whether this attribute has a true or false value, it
27513 b1d1fe7fb514 Document the demise of all pretenses of safety, and the difference Guido van Rossum <guido@python.org> parents: 27507 diff changeset	1614 only matters whether it exists (XXX this is a bug; cPickle
b1d1fe7fb514 Document the demise of all pretenses of safety, and the difference Guido van Rossum <guido@python.org> parents: 27507 diff changeset	1615 requires the attribute to be true). If __safe_for_unpickling__
b1d1fe7fb514 Document the demise of all pretenses of safety, and the difference Guido van Rossum <guido@python.org> parents: 27507 diff changeset	1616 doesn't exist, UnpicklingError is raised.
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1617
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1618 Else (the class object does have a __safe_for_unpickling__ attr),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1619 the class object obtained from INST's arguments is applied to the
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1620 argtuple obtained from the stack, and the resulting instance object
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1621 is pushed on the stack.
27535 72b38d5fe61f Updated the INST/OBJ docs, to say they really are (almost) identical Tim Peters <tim.peters@gmail.com> parents: 27532 diff changeset	1622
72b38d5fe61f Updated the INST/OBJ docs, to say they really are (almost) identical Tim Peters <tim.peters@gmail.com> parents: 27532 diff changeset	1623 NOTE: checks for __safe_for_unpickling__ went away in Python 2.3.
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1624 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1625
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1626 I(name='OBJ',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1627 code='o',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1628 arg=None,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1629 stack_before=[markobject, anyobject, stackslice],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1630 stack_after=[anyobject],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1631 proto=1,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1632 doc="""Build a class instance.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1633
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1634 This is the protocol 1 version of protocol 0's INST opcode, and is
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1635 very much like it. The major difference is that the class object
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1636 is taken off the stack, allowing it to be retrieved from the memo
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1637 repeatedly if several instances of the same class are created. This
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1638 can be much more efficient (in both time and space) than repeatedly
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1639 embedding the module and class names in INST opcodes.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1640
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1641 Unlike INST, OBJ takes no arguments from the opcode stream. Instead
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1642 the class object is taken off the stack, immediately above the
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1643 topmost markobject:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1644
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1645 Stack before: ... markobject classobject stackslice
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1646 Stack after: ... new_instance_object
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1647
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1648 As for INST, the remainder of the stack above the markobject is
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1649 gathered into an argument tuple, and then the logic seems identical,
27513 b1d1fe7fb514 Document the demise of all pretenses of safety, and the difference Guido van Rossum <guido@python.org> parents: 27507 diff changeset	1650 except that no __safe_for_unpickling__ check is done (XXX this is
b1d1fe7fb514 Document the demise of all pretenses of safety, and the difference Guido van Rossum <guido@python.org> parents: 27507 diff changeset	1651 a bug; cPickle does test __safe_for_unpickling__). See INST for
b1d1fe7fb514 Document the demise of all pretenses of safety, and the difference Guido van Rossum <guido@python.org> parents: 27507 diff changeset	1652 the gory details.
27535 72b38d5fe61f Updated the INST/OBJ docs, to say they really are (almost) identical Tim Peters <tim.peters@gmail.com> parents: 27532 diff changeset	1653
72b38d5fe61f Updated the INST/OBJ docs, to say they really are (almost) identical Tim Peters <tim.peters@gmail.com> parents: 27532 diff changeset	1654 NOTE: In Python 2.3, INST and OBJ are identical except for how they
72b38d5fe61f Updated the INST/OBJ docs, to say they really are (almost) identical Tim Peters <tim.peters@gmail.com> parents: 27532 diff changeset	1655 get the class object. That was always the intent; the implementations
72b38d5fe61f Updated the INST/OBJ docs, to say they really are (almost) identical Tim Peters <tim.peters@gmail.com> parents: 27532 diff changeset	1656 had diverged for accidental reasons.
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1657 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1658
27462 26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1659 I(name='NEWOBJ',
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1660 code='\x81',
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1661 arg=None,
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1662 stack_before=[anyobject, anyobject],
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1663 stack_after=[anyobject],
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1664 proto=2,
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1665 doc="""Build an object instance.
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1666
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1667 The stack before should be thought of as containing a class
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1668 object followed by an argument tuple (the tuple being the stack
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1669 top). Call these cls and args. They are popped off the stack,
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1670 and the value returned by cls.__new__(cls, *args) is pushed back
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1671 onto the stack.
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1672 """),
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1673
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1674 # Machine control.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1675
27462 26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1676 I(name='PROTO',
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1677 code='\x80',
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1678 arg=uint1,
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1679 stack_before=[],
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1680 stack_after=[],
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1681 proto=2,
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1682 doc="""Protocol version indicator.
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1683
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1684 For protocol 2 and above, a pickle must start with this opcode.
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1685 The argument is the protocol version, an int in range(2, 256).
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1686 """),
26a792007665 Now that proto2 is defined, replaced XXX blocks with text about it. Tim Peters <tim.peters@gmail.com> parents: 27436 diff changeset	1687
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1688 I(name='STOP',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1689 code='.',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1690 arg=None,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1691 stack_before=[anyobject],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1692 stack_after=[],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1693 proto=0,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1694 doc="""Stop the unpickling machine.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1695
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1696 Every pickle ends with this opcode. The object at the top of the stack
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1697 is popped, and that's the result of unpickling. The stack should be
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1698 empty then.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1699 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1700
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1701 # Ways to deal with persistent IDs.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1702
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1703 I(name='PERSID',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1704 code='P',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1705 arg=stringnl_noescape,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1706 stack_before=[],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1707 stack_after=[anyobject],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1708 proto=0,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1709 doc="""Push an object identified by a persistent ID.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1710
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1711 The pickle module doesn't define what a persistent ID means. PERSID's
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1712 argument is a newline-terminated str-style (no embedded escapes, no
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1713 bracketing quote characters) string, which is "the persistent ID".
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1714 The unpickler passes this string to self.persistent_load(). Whatever
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1715 object that returns is pushed on the stack. There is no implementation
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1716 of persistent_load() in Python's unpickler: it must be supplied by an
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1717 unpickler subclass.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1718 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1719
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1720 I(name='BINPERSID',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1721 code='Q',
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1722 arg=None,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1723 stack_before=[anyobject],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1724 stack_after=[anyobject],
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1725 proto=1,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1726 doc="""Push an object identified by a persistent ID.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1727
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1728 Like PERSID, except the persistent ID is popped off the stack (instead
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1729 of being a string embedded in the opcode bytestream). The persistent
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1730 ID is passed to self.persistent_load(), and whatever object that
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1731 returns is pushed on the stack. See PERSID for more detail.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1732 """),
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1733 ]
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1734 del I
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1735
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1736 # Verify uniqueness of .name and .code members.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1737 name2i = {}
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1738 code2i = {}
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1739
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1740 for i, d in enumerate(opcodes):
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1741 if d.name in name2i:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1742 raise ValueError("repeated name %r at indices %d and %d" %
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1743 (d.name, name2i[d.name], i))
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1744 if d.code in code2i:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1745 raise ValueError("repeated code %r at indices %d and %d" %
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1746 (d.code, code2i[d.code], i))
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1747
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1748 name2i[d.name] = i
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1749 code2i[d.code] = i
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1750
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1751 del name2i, code2i, i, d
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1752
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1753 ##############################################################################
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1754 # Build a code2op dict, mapping opcode characters to OpcodeInfo records.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1755 # Also ensure we've got the same stuff as pickle.py, although the
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1756 # introspection here is dicey.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1757
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1758 code2op = {}
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1759 for d in opcodes:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1760 code2op[d.code] = d
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1761 del d
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1762
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1763 def assure_pickle_consistency(verbose=False):
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1764 import pickle, re
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1765
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1766 copy = code2op.copy()
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1767 for name in pickle.__all__:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1768 if not re.match("[A-Z][A-Z0-9_]+$", name):
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1769 if verbose:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1770 print "skipping %r: it doesn't look like an opcode name" % name
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1771 continue
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1772 picklecode = getattr(pickle, name)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1773 if not isinstance(picklecode, str) or len(picklecode) != 1:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1774 if verbose:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1775 print ("skipping %r: value %r doesn't look like a pickle "
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1776 "code" % (name, picklecode))
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1777 continue
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1778 if picklecode in copy:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1779 if verbose:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1780 print "checking name %r w/ code %r for consistency" % (
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1781 name, picklecode)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1782 d = copy[picklecode]
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1783 if d.name != name:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1784 raise ValueError("for pickle code %r, pickle.py uses name %r "
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1785 "but we're using name %r" % (picklecode,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1786 name,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1787 d.name))
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1788 # Forget this one. Any left over in copy at the end are a problem
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1789 # of a different kind.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1790 del copy[picklecode]
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1791 else:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1792 raise ValueError("pickle.py appears to have a pickle opcode with "
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1793 "name %r and code %r, but we don't" %
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1794 (name, picklecode))
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1795 if copy:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1796 msg = ["we appear to have pickle opcodes that pickle.py doesn't have:"]
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1797 for code, d in copy.items():
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1798 msg.append(" name %r with code %r" % (d.name, code))
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1799 raise ValueError("\n".join(msg))
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1800
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1801 assure_pickle_consistency()
27506 b480a4f707f6 pickle: Comment repair. Tim Peters <tim.peters@gmail.com> parents: 27473 diff changeset	1802 del assure_pickle_consistency
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1803
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1804 ##############################################################################
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1805 # A pickle opcode generator.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1806
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1807 def genops(pickle):
27424 93ba42cf4e53 Remove a stray quote. Guido van Rossum <guido@python.org> parents: 27423 diff changeset	1808 """Generate all the opcodes in a pickle.
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1809
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1810 'pickle' is a file-like object, or string, containing the pickle.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1811
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1812 Each opcode in the pickle is generated, from the current pickle position,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1813 stopping after a STOP opcode is delivered. A triple is generated for
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1814 each opcode:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1815
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1816 opcode, arg, pos
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1817
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1818 opcode is an OpcodeInfo record, describing the current opcode.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1819
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1820 If the opcode has an argument embedded in the pickle, arg is its decoded
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1821 value, as a Python object. If the opcode doesn't have an argument, arg
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1822 is None.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1823
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1824 If the pickle has a tell() method, pos was the value of pickle.tell()
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1825 before reading the current opcode. If the pickle is a string object,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1826 it's wrapped in a StringIO object, and the latter's tell() result is
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1827 used. Else (the pickle doesn't have a tell(), and it's not obvious how
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1828 to query its current position) pos is None.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1829 """
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1830
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1831 import cStringIO as StringIO
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1832
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1833 if isinstance(pickle, str):
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1834 pickle = StringIO.StringIO(pickle)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1835
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1836 if hasattr(pickle, "tell"):
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1837 getpos = pickle.tell
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1838 else:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1839 getpos = lambda: None
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1840
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1841 while True:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1842 pos = getpos()
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1843 code = pickle.read(1)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1844 opcode = code2op.get(code)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1845 if opcode is None:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1846 if code == "":
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1847 raise ValueError("pickle exhausted before seeing STOP")
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1848 else:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1849 raise ValueError("at position %s, opcode %r unknown" % (
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1850 pos is None and "<unknown>" or pos,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1851 code))
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1852 if opcode.arg is None:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1853 arg = None
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1854 else:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1855 arg = opcode.arg.reader(pickle)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1856 yield opcode, arg, pos
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1857 if code == '.':
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1858 assert opcode.name == 'STOP'
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1859 break
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1860
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1861 ##############################################################################
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1862 # A symbolic pickle disassembler.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1863
27703 0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	1864 def dis(pickle, out=None, memo=None, indentlevel=4):
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1865 """Produce a symbolic disassembly of a pickle.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1866
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1867 'pickle' is a file-like object, or string, containing a (at least one)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1868 pickle. The pickle is disassembled from the current position, through
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1869 the first STOP opcode encountered.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1870
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1871 Optional arg 'out' is a file-like object to which the disassembly is
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1872 printed. It defaults to sys.stdout.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1873
27703 0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	1874 Optional arg 'memo' is a Python dict, used as the pickle's memo. It
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	1875 may be mutated by dis(), if the pickle contains PUT or BINPUT opcodes.
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	1876 Passing the same memo object to another dis() call then allows disassembly
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	1877 to proceed across multiple pickles that were all created by the same
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	1878 pickler with the same memo. Ordinarily you don't need to worry about this.
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	1879
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1880 Optional arg indentlevel is the number of blanks by which to indent
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1881 a new MARK level. It defaults to 4.
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1882
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1883 In addition to printing the disassembly, some sanity checks are made:
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1884
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1885 + All embedded opcode arguments "make sense".
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1886
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1887 + Explicit and implicit pop operations have enough items on the stack.
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1888
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1889 + When an opcode implicitly refers to a markobject, a markobject is
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1890 actually on the stack.
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1891
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1892 + A memo entry isn't referenced before it's defined.
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1893
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1894 + The markobject isn't stored in the memo.
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1895
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1896 + A memo entry isn't redefined.
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1897 """
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1898
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1899 # Most of the hair here is for sanity checks, but most of it is needed
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1900 # anyway to detect when a protocol 0 POP takes a MARK off the stack
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1901 # (which in turn is needed to indent MARK blocks correctly).
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1902
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1903 stack = [] # crude emulation of unpickler stack
27703 0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	1904 if memo is None:
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	1905 memo = {} # crude emulation of unpicker memo
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1906 maxproto = -1 # max protocol number seen
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1907 markstack = [] # bytecode positions of MARK opcodes
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1908 indentchunk = ' ' * indentlevel
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1909 errormsg = None
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1910 for opcode, arg, pos in genops(pickle):
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1911 if pos is not None:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1912 print >> out, "%5d:" % pos,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1913
27471 05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	1914 line = "%-4s %s%s" % (repr(opcode.code)[1:-1],
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	1915 indentchunk * len(markstack),
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	1916 opcode.name)
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1917
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1918 maxproto = max(maxproto, opcode.proto)
27532 02f138c639a0 dis(): Simplified stack emulation a bit. Tim Peters <tim.peters@gmail.com> parents: 27522 diff changeset	1919 before = opcode.stack_before # don't mutate
02f138c639a0 dis(): Simplified stack emulation a bit. Tim Peters <tim.peters@gmail.com> parents: 27522 diff changeset	1920 after = opcode.stack_after # don't mutate
02f138c639a0 dis(): Simplified stack emulation a bit. Tim Peters <tim.peters@gmail.com> parents: 27522 diff changeset	1921 numtopop = len(before)
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1922
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1923 # See whether a MARK should be popped.
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1924 markmsg = None
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1925 if markobject in before or (opcode.name == "POP" and
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1926 stack and
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1927 stack[-1] is markobject):
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1928 assert markobject not in after
27532 02f138c639a0 dis(): Simplified stack emulation a bit. Tim Peters <tim.peters@gmail.com> parents: 27522 diff changeset	1929 if __debug__:
02f138c639a0 dis(): Simplified stack emulation a bit. Tim Peters <tim.peters@gmail.com> parents: 27522 diff changeset	1930 if markobject in before:
02f138c639a0 dis(): Simplified stack emulation a bit. Tim Peters <tim.peters@gmail.com> parents: 27522 diff changeset	1931 assert before[-1] is stackslice
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1932 if markstack:
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1933 markpos = markstack.pop()
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1934 if markpos is None:
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1935 markmsg = "(MARK at unknown opcode offset)"
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1936 else:
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1937 markmsg = "(MARK at %d)" % markpos
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1938 # Pop everything at and after the topmost markobject.
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1939 while stack[-1] is not markobject:
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1940 stack.pop()
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1941 stack.pop()
27532 02f138c639a0 dis(): Simplified stack emulation a bit. Tim Peters <tim.peters@gmail.com> parents: 27522 diff changeset	1942 # Stop later code from popping too much.
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1943 try:
27532 02f138c639a0 dis(): Simplified stack emulation a bit. Tim Peters <tim.peters@gmail.com> parents: 27522 diff changeset	1944 numtopop = before.index(markobject)
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1945 except ValueError:
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1946 assert opcode.name == "POP"
27532 02f138c639a0 dis(): Simplified stack emulation a bit. Tim Peters <tim.peters@gmail.com> parents: 27522 diff changeset	1947 numtopop = 0
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1948 else:
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1949 errormsg = markmsg = "no MARK exists on stack"
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1950
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1951 # Check for correct memo usage.
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1952 if opcode.name in ("PUT", "BINPUT", "LONG_BINPUT"):
27532 02f138c639a0 dis(): Simplified stack emulation a bit. Tim Peters <tim.peters@gmail.com> parents: 27522 diff changeset	1953 assert arg is not None
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1954 if arg in memo:
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1955 errormsg = "memo key %r already defined" % arg
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1956 elif not stack:
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1957 errormsg = "stack is empty -- can't store into memo"
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1958 elif stack[-1] is markobject:
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1959 errormsg = "can't store markobject in the memo"
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1960 else:
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1961 memo[arg] = stack[-1]
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1962
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1963 elif opcode.name in ("GET", "BINGET", "LONG_BINGET"):
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1964 if arg in memo:
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1965 assert len(after) == 1
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1966 after = [memo[arg]] # for better stack emulation
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1967 else:
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1968 errormsg = "memo key %r has never been stored into" % arg
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1969
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1970 if arg is not None or markmsg:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1971 # make a mild effort to align arguments
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1972 line += ' ' * (10 - len(opcode.name))
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1973 if arg is not None:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1974 line += ' ' + repr(arg)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1975 if markmsg:
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1976 line += ' ' + markmsg
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1977 print >> out, line
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1978
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1979 if errormsg:
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1980 # Note that we delayed complaining until the offending opcode
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1981 # was printed.
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1982 raise ValueError(errormsg)
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1983
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1984 # Emulate the stack effects.
27532 02f138c639a0 dis(): Simplified stack emulation a bit. Tim Peters <tim.peters@gmail.com> parents: 27522 diff changeset	1985 if len(stack) < numtopop:
02f138c639a0 dis(): Simplified stack emulation a bit. Tim Peters <tim.peters@gmail.com> parents: 27522 diff changeset	1986 raise ValueError("tries to pop %d items from stack with "
02f138c639a0 dis(): Simplified stack emulation a bit. Tim Peters <tim.peters@gmail.com> parents: 27522 diff changeset	1987 "only %d items" % (numtopop, len(stack)))
02f138c639a0 dis(): Simplified stack emulation a bit. Tim Peters <tim.peters@gmail.com> parents: 27522 diff changeset	1988 if numtopop:
02f138c639a0 dis(): Simplified stack emulation a bit. Tim Peters <tim.peters@gmail.com> parents: 27522 diff changeset	1989 del stack[-numtopop:]
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1990 if markobject in after:
27532 02f138c639a0 dis(): Simplified stack emulation a bit. Tim Peters <tim.peters@gmail.com> parents: 27522 diff changeset	1991 assert markobject not in before
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1992 markstack.append(pos)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1993
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1994 stack.extend(after)
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1995
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1996 print >> out, "highest protocol among opcodes =", maxproto
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1997 if stack:
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	1998 raise ValueError("stack not empty after STOP: %r" % stack)
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	1999
27472 de4061584c83 _dis_test should be a raw string now that it contains \x escapes. Guido van Rossum <guido@python.org> parents: 27471 diff changeset	2000 _dis_test = r"""
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2001 >>> import pickle
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2002 >>> x = [1, 2, (3, 4), {'abc': u"def"}]
27468 6ffafcef98fb Fix one disassembly output now that empty tuples are no longer Guido van Rossum <guido@python.org> parents: 27463 diff changeset	2003 >>> pkl = pickle.dumps(x, 0)
6ffafcef98fb Fix one disassembly output now that empty tuples are no longer Guido van Rossum <guido@python.org> parents: 27463 diff changeset	2004 >>> dis(pkl)
27471 05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2005 0: ( MARK
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2006 1: l LIST (MARK at 0)
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2007 2: p PUT 0
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2008 5: I INT 1
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2009 8: a APPEND
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2010 9: I INT 2
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2011 12: a APPEND
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2012 13: ( MARK
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2013 14: I INT 3
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2014 17: I INT 4
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2015 20: t TUPLE (MARK at 13)
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2016 21: p PUT 1
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2017 24: a APPEND
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2018 25: ( MARK
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2019 26: d DICT (MARK at 25)
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2020 27: p PUT 2
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2021 30: S STRING 'abc'
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2022 37: p PUT 3
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2023 40: V UNICODE u'def'
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2024 45: p PUT 4
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2025 48: s SETITEM
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2026 49: a APPEND
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2027 50: . STOP
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	2028 highest protocol among opcodes = 0
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2029
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2030 Try again with a "binary" pickle.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2031
27468 6ffafcef98fb Fix one disassembly output now that empty tuples are no longer Guido van Rossum <guido@python.org> parents: 27463 diff changeset	2032 >>> pkl = pickle.dumps(x, 1)
6ffafcef98fb Fix one disassembly output now that empty tuples are no longer Guido van Rossum <guido@python.org> parents: 27463 diff changeset	2033 >>> dis(pkl)
27471 05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2034 0: ] EMPTY_LIST
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2035 1: q BINPUT 0
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2036 3: ( MARK
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2037 4: K BININT1 1
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2038 6: K BININT1 2
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2039 8: ( MARK
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2040 9: K BININT1 3
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2041 11: K BININT1 4
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2042 13: t TUPLE (MARK at 8)
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2043 14: q BINPUT 1
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2044 16: } EMPTY_DICT
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2045 17: q BINPUT 2
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2046 19: U SHORT_BINSTRING 'abc'
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2047 24: q BINPUT 3
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2048 26: X BINUNICODE u'def'
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2049 34: q BINPUT 4
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2050 36: s SETITEM
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2051 37: e APPENDS (MARK at 3)
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2052 38: . STOP
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	2053 highest protocol among opcodes = 1
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2054
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2055 Exercise the INST/OBJ/BUILD family.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2056
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2057 >>> import random
27431 f8a90085acc4 Begin the change from 'binary vs. text mode' to 'protocol 0, 1, 2'. Guido van Rossum <guido@python.org> parents: 27429 diff changeset	2058 >>> dis(pickle.dumps(random.random, 0))
27471 05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2059 0: c GLOBAL 'random random'
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2060 15: p PUT 0
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2061 18: . STOP
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	2062 highest protocol among opcodes = 0
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2063
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2064 >>> x = [pickle.PicklingError()] * 2
27431 f8a90085acc4 Begin the change from 'binary vs. text mode' to 'protocol 0, 1, 2'. Guido van Rossum <guido@python.org> parents: 27429 diff changeset	2065 >>> dis(pickle.dumps(x, 0))
27471 05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2066 0: ( MARK
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2067 1: l LIST (MARK at 0)
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2068 2: p PUT 0
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2069 5: ( MARK
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2070 6: i INST 'pickle PicklingError' (MARK at 5)
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2071 28: p PUT 1
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2072 31: ( MARK
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2073 32: d DICT (MARK at 31)
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2074 33: p PUT 2
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2075 36: S STRING 'args'
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2076 44: p PUT 3
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2077 47: ( MARK
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2078 48: t TUPLE (MARK at 47)
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2079 49: s SETITEM
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2080 50: b BUILD
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2081 51: a APPEND
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2082 52: g GET 1
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2083 55: a APPEND
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2084 56: . STOP
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	2085 highest protocol among opcodes = 0
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2086
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2087 >>> dis(pickle.dumps(x, 1))
27471 05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2088 0: ] EMPTY_LIST
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2089 1: q BINPUT 0
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2090 3: ( MARK
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2091 4: ( MARK
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2092 5: c GLOBAL 'pickle PicklingError'
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2093 27: q BINPUT 1
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2094 29: o OBJ (MARK at 4)
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2095 30: q BINPUT 2
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2096 32: } EMPTY_DICT
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2097 33: q BINPUT 3
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2098 35: U SHORT_BINSTRING 'args'
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2099 41: q BINPUT 4
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2100 43: ) EMPTY_TUPLE
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2101 44: s SETITEM
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2102 45: b BUILD
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2103 46: h BINGET 2
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2104 48: e APPENDS (MARK at 3)
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2105 49: . STOP
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	2106 highest protocol among opcodes = 1
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2107
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2108 Try "the canonical" recursive-object test.
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2109
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2110 >>> L = []
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2111 >>> T = L,
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2112 >>> L.append(T)
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2113 >>> L[0] is T
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2114 True
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2115 >>> T[0] is L
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2116 True
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2117 >>> L[0][0] is L
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2118 True
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2119 >>> T[0][0] is T
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2120 True
27431 f8a90085acc4 Begin the change from 'binary vs. text mode' to 'protocol 0, 1, 2'. Guido van Rossum <guido@python.org> parents: 27429 diff changeset	2121 >>> dis(pickle.dumps(L, 0))
27471 05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2122 0: ( MARK
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2123 1: l LIST (MARK at 0)
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2124 2: p PUT 0
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2125 5: ( MARK
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2126 6: g GET 0
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2127 9: t TUPLE (MARK at 5)
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2128 10: p PUT 1
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2129 13: a APPEND
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2130 14: . STOP
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	2131 highest protocol among opcodes = 0
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	2132
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2133 >>> dis(pickle.dumps(L, 1))
27471 05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2134 0: ] EMPTY_LIST
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2135 1: q BINPUT 0
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2136 3: ( MARK
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2137 4: h BINGET 0
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2138 6: t TUPLE (MARK at 3)
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2139 7: q BINPUT 1
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2140 9: a APPEND
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2141 10: . STOP
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	2142 highest protocol among opcodes = 1
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2143
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	2144 Note that, in the protocol 0 pickle of the recursive tuple, the disassembler
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	2145 has to emulate the stack in order to realize that the POP opcode at 16 gets
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	2146 rid of the MARK at 0.
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2147
27431 f8a90085acc4 Begin the change from 'binary vs. text mode' to 'protocol 0, 1, 2'. Guido van Rossum <guido@python.org> parents: 27429 diff changeset	2148 >>> dis(pickle.dumps(T, 0))
27471 05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2149 0: ( MARK
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2150 1: ( MARK
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2151 2: l LIST (MARK at 1)
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2152 3: p PUT 0
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2153 6: ( MARK
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2154 7: g GET 0
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2155 10: t TUPLE (MARK at 6)
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2156 11: p PUT 1
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2157 14: a APPEND
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2158 15: 0 POP
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	2159 16: 0 POP (MARK at 0)
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	2160 17: g GET 1
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	2161 20: . STOP
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	2162 highest protocol among opcodes = 0
0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	2163
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2164 >>> dis(pickle.dumps(T, 1))
27471 05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2165 0: ( MARK
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2166 1: ] EMPTY_LIST
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2167 2: q BINPUT 0
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2168 4: ( MARK
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2169 5: h BINGET 0
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2170 7: t TUPLE (MARK at 4)
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2171 8: q BINPUT 1
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2172 10: a APPEND
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2173 11: 1 POP_MARK (MARK at 0)
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2174 12: h BINGET 1
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2175 14: . STOP
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	2176 highest protocol among opcodes = 1
27471 05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2177
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2178 Try protocol 2.
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2179
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2180 >>> dis(pickle.dumps(L, 2))
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2181 0: \x80 PROTO 2
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2182 2: ] EMPTY_LIST
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2183 3: q BINPUT 0
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2184 5: h BINGET 0
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2185 7: \x85 TUPLE1
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2186 8: q BINPUT 1
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2187 10: a APPEND
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2188 11: . STOP
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	2189 highest protocol among opcodes = 2
27471 05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2190
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2191 >>> dis(pickle.dumps(T, 2))
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2192 0: \x80 PROTO 2
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2193 2: ] EMPTY_LIST
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2194 3: q BINPUT 0
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2195 5: h BINGET 0
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2196 7: \x85 TUPLE1
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2197 8: q BINPUT 1
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2198 10: a APPEND
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2199 11: 0 POP
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2200 12: h BINGET 1
05f7e1c65d7e dis(): Not all opcodes are printable anymore, so print the repr Tim Peters <tim.peters@gmail.com> parents: 27468 diff changeset	2201 14: . STOP
27522 0f9eb8413b61 dis(): This had a problem with proto 0 pickles, in that POP sometimes Tim Peters <tim.peters@gmail.com> parents: 27518 diff changeset	2202 highest protocol among opcodes = 2
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2203 """
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2204
27703 0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	2205 _memo_test = r"""
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	2206 >>> import pickle
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	2207 >>> from StringIO import StringIO
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	2208 >>> f = StringIO()
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	2209 >>> p = pickle.Pickler(f, 2)
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	2210 >>> x = [1, 2, 3]
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	2211 >>> p.dump(x)
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	2212 >>> p.dump(x)
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	2213 >>> f.seek(0)
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	2214 >>> memo = {}
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	2215 >>> dis(f, memo=memo)
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	2216 0: \x80 PROTO 2
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	2217 2: ] EMPTY_LIST
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	2218 3: q BINPUT 0
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	2219 5: ( MARK
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	2220 6: K BININT1 1
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	2221 8: K BININT1 2
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	2222 10: K BININT1 3
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	2223 12: e APPENDS (MARK at 5)
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	2224 13: . STOP
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	2225 highest protocol among opcodes = 2
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	2226 >>> dis(f, memo=memo)
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	2227 14: \x80 PROTO 2
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	2228 16: h BINGET 0
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	2229 18: . STOP
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	2230 highest protocol among opcodes = 2
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	2231 """
0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	2232
27468 6ffafcef98fb Fix one disassembly output now that empty tuples are no longer Guido van Rossum <guido@python.org> parents: 27463 diff changeset	2233 __test__ = {'disassembler_test': _dis_test,
27703 0428d59069a9 dis(): Added an optional memo argument, so that multiple pickles in a Tim Peters <tim.peters@gmail.com> parents: 27552 diff changeset	2234 'disassembler_memo_test': _memo_test,
27421 0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2235 }
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2236
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2237 def _test():
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2238 import doctest
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2239 return doctest.testmod()
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2240
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2241 if __name__ == "__main__":
0aeaa3d1e0ff Moving pickletools.py from the sandbox into the std library. I started Tim Peters <tim.peters@gmail.com> parents: diff changeset	2242 _test()

Mercurial > cpython-lru_cache-weakref

annotate Lib/pickletools.py @ 27998:e27f58a50737 legacy-trunk