💾 Archived View for tris.fyi › pydoc › pickletools captured on 2022-01-08 at 13:41:44. Gemini links have been rewritten to link to archived content
-=-=-=-=-=-=-
"Executable documentation" for the pickle module. Extensive comments about the pickle protocols and pickle-machine opcodes can be found here. Some functions meant for external use: genops(pickle) Generate all the opcodes in a pickle, as (opcode, arg, position) triples. dis(pickle, out=None, memo=None, indentlevel=4) Print a symbolic disassembly of a pickle.
doc = <member 'doc' of 'ArgumentDescriptor' objects>
n = <member 'n' of 'ArgumentDescriptor' objects>
name = <member 'name' of 'ArgumentDescriptor' objects>
reader = <member 'reader' of 'ArgumentDescriptor' objects>
arg = <member 'arg' of 'OpcodeInfo' objects>
code = <member 'code' of 'OpcodeInfo' objects>
doc = <member 'doc' of 'OpcodeInfo' objects>
name = <member 'name' of 'OpcodeInfo' objects>
proto = <member 'proto' of 'OpcodeInfo' objects>
stack_after = <member 'stack_after' of 'OpcodeInfo' objects>
stack_before = <member 'stack_before' of 'OpcodeInfo' objects>
doc = <member 'doc' of 'StackObject' objects>
name = <member 'name' of 'StackObject' objects>
obtype = <member 'obtype' of 'StackObject' objects>
decode_long(data)
Decode a long from a two's complement little-endian binary string.
>>> decode_long(b'')
0
>>> decode_long(b"\xff\x00")
255
>>> decode_long(b"\xff\x7f")
32767
>>> decode_long(b"\x00\xff")
-256
>>> decode_long(b"\x00\x80")
-32768
>>> decode_long(b"\x80")
-128
>>> decode_long(b"\x7f")
127
dis(pickle, out=None, memo=None, indentlevel=4, annotate=0)
Produce a symbolic disassembly of a pickle.
'pickle' is a file-like object, or string, containing a (at least one)
pickle. The pickle is disassembled from the current position, through
the first STOP opcode encountered.
Optional arg 'out' is a file-like object to which the disassembly is
printed. It defaults to sys.stdout.
Optional arg 'memo' is a Python dict, used as the pickle's memo. It
may be mutated by dis(), if the pickle contains PUT or BINPUT opcodes.
Passing the same memo object to another dis() call then allows disassembly
to proceed across multiple pickles that were all created by the same
pickler with the same memo. Ordinarily you don't need to worry about this.
Optional arg 'indentlevel' is the number of blanks by which to indent
a new MARK level. It defaults to 4.
Optional arg 'annotate' if nonzero instructs dis() to add short
description of the opcode on each line of disassembled output.
The value given to 'annotate' must be an integer and is used as a
hint for the column where annotation should start. The default
value is 0, meaning no annotations.
In addition to printing the disassembly, some sanity checks are made:
+ All embedded opcode arguments "make sense".
+ Explicit and implicit pop operations have enough items on the stack.
+ When an opcode implicitly refers to a markobject, a markobject is
actually on the stack.
+ A memo entry isn't referenced before it's defined.
+ The markobject isn't stored in the memo.
+ A memo entry isn't redefined.
genops(pickle)
Generate all the opcodes in a pickle.
'pickle' is a file-like object, or string, containing the pickle.
Each opcode in the pickle is generated, from the current pickle position,
stopping after a STOP opcode is delivered. A triple is generated for
each opcode:
opcode, arg, pos
opcode is an OpcodeInfo record, describing the current opcode.
If the opcode has an argument embedded in the pickle, arg is its decoded
value, as a Python object. If the opcode doesn't have an argument, arg
is None.
If the pickle has a tell() method, pos was the value of pickle.tell()
before reading the current opcode. If the pickle is a bytes object,
it's wrapped in a BytesIO object, and the latter's tell() result is
used. Else (the pickle doesn't have a tell(), and it's not obvious how
to query its current position) pos is None.
optimize(p) Optimize a pickle string by removing unused PUT opcodes
read_bytearray8(f)
>>> import io, struct, sys
>>> read_bytearray8(io.BytesIO(b"\x00\x00\x00\x00\x00\x00\x00\x00abc"))
bytearray(b'')
>>> read_bytearray8(io.BytesIO(b"\x03\x00\x00\x00\x00\x00\x00\x00abcdef"))
bytearray(b'abc')
>>> bigsize8 = struct.pack("<Q", sys.maxsize//3)
>>> read_bytearray8(io.BytesIO(bigsize8 + b"abcdef")) #doctest: +ELLIPSIS
Traceback (most recent call last):
...
ValueError: expected ... bytes in a bytearray8, but only 6 remain
read_bytes1(f)
>>> import io
>>> read_bytes1(io.BytesIO(b"\x00"))
b''
>>> read_bytes1(io.BytesIO(b"\x03abcdef"))
b'abc'
read_bytes4(f)
>>> import io
>>> read_bytes4(io.BytesIO(b"\x00\x00\x00\x00abc"))
b''
>>> read_bytes4(io.BytesIO(b"\x03\x00\x00\x00abcdef"))
b'abc'
>>> read_bytes4(io.BytesIO(b"\x00\x00\x00\x03abcdef"))
Traceback (most recent call last):
...
ValueError: expected 50331648 bytes in a bytes4, but only 6 remain
read_bytes8(f)
>>> import io, struct, sys
>>> read_bytes8(io.BytesIO(b"\x00\x00\x00\x00\x00\x00\x00\x00abc"))
b''
>>> read_bytes8(io.BytesIO(b"\x03\x00\x00\x00\x00\x00\x00\x00abcdef"))
b'abc'
>>> bigsize8 = struct.pack("<Q", sys.maxsize//3)
>>> read_bytes8(io.BytesIO(bigsize8 + b"abcdef")) #doctest: +ELLIPSIS
Traceback (most recent call last):
...
ValueError: expected ... bytes in a bytes8, but only 6 remain
read_decimalnl_long(f)
>>> import io
>>> read_decimalnl_long(io.BytesIO(b"1234L\n56"))
1234
>>> read_decimalnl_long(io.BytesIO(b"123456789012345678901234L\n6"))
123456789012345678901234
read_decimalnl_short(f)
>>> import io
>>> read_decimalnl_short(io.BytesIO(b"1234\n56"))
1234
>>> read_decimalnl_short(io.BytesIO(b"1234L\n56"))
Traceback (most recent call last):
...
ValueError: invalid literal for int() with base 10: b'1234L'
read_float8(f)
>>> import io, struct
>>> raw = struct.pack(">d", -1.25)
>>> raw
b'\xbf\xf4\x00\x00\x00\x00\x00\x00'
>>> read_float8(io.BytesIO(raw + b"\n"))
-1.25
read_floatnl(f)
>>> import io
>>> read_floatnl(io.BytesIO(b"-1.25\n6"))
-1.25
read_int4(f)
>>> import io
>>> read_int4(io.BytesIO(b'\xff\x00\x00\x00'))
255
>>> read_int4(io.BytesIO(b'\x00\x00\x00\x80')) == -(2**31)
True
read_long1(f)
>>> import io
>>> read_long1(io.BytesIO(b"\x00"))
0
>>> read_long1(io.BytesIO(b"\x02\xff\x00"))
255
>>> read_long1(io.BytesIO(b"\x02\xff\x7f"))
32767
>>> read_long1(io.BytesIO(b"\x02\x00\xff"))
-256
>>> read_long1(io.BytesIO(b"\x02\x00\x80"))
-32768
read_long4(f)
>>> import io
>>> read_long4(io.BytesIO(b"\x02\x00\x00\x00\xff\x00"))
255
>>> read_long4(io.BytesIO(b"\x02\x00\x00\x00\xff\x7f"))
32767
>>> read_long4(io.BytesIO(b"\x02\x00\x00\x00\x00\xff"))
-256
>>> read_long4(io.BytesIO(b"\x02\x00\x00\x00\x00\x80"))
-32768
>>> read_long1(io.BytesIO(b"\x00\x00\x00\x00"))
0
read_string1(f)
>>> import io
>>> read_string1(io.BytesIO(b"\x00"))
''
>>> read_string1(io.BytesIO(b"\x03abcdef"))
'abc'
read_string4(f)
>>> import io
>>> read_string4(io.BytesIO(b"\x00\x00\x00\x00abc"))
''
>>> read_string4(io.BytesIO(b"\x03\x00\x00\x00abcdef"))
'abc'
>>> read_string4(io.BytesIO(b"\x00\x00\x00\x03abcdef"))
Traceback (most recent call last):
...
ValueError: expected 50331648 bytes in a string4, but only 6 remain
read_stringnl(f, decode=True, stripquotes=True)
>>> import io
>>> read_stringnl(io.BytesIO(b"'abcd'\nefg\n"))
'abcd'
>>> read_stringnl(io.BytesIO(b"\n"))
Traceback (most recent call last):
...
ValueError: no string quotes around b''
>>> read_stringnl(io.BytesIO(b"\n"), stripquotes=False)
''
>>> read_stringnl(io.BytesIO(b"''\n"))
''
>>> read_stringnl(io.BytesIO(b'"abcd"'))
Traceback (most recent call last):
...
ValueError: no newline found when trying to read stringnl
Embedded escapes are undone in the result.
>>> read_stringnl(io.BytesIO(br"'a\n\\b\x00c\td'" + b"\n'e'"))
'a\n\\b\x00c\td'
read_stringnl_noescape(f)
read_stringnl_noescape_pair(f)
>>> import io
>>> read_stringnl_noescape_pair(io.BytesIO(b"Queue\nEmpty\njunk"))
'Queue Empty'
read_uint1(f)
>>> import io
>>> read_uint1(io.BytesIO(b'\xff'))
255
read_uint2(f)
>>> import io
>>> read_uint2(io.BytesIO(b'\xff\x00'))
255
>>> read_uint2(io.BytesIO(b'\xff\xff'))
65535
read_uint4(f)
>>> import io
>>> read_uint4(io.BytesIO(b'\xff\x00\x00\x00'))
255
>>> read_uint4(io.BytesIO(b'\x00\x00\x00\x80')) == 2**31
True
read_uint8(f)
>>> import io
>>> read_uint8(io.BytesIO(b'\xff\x00\x00\x00\x00\x00\x00\x00'))
255
>>> read_uint8(io.BytesIO(b'\xff' * 8)) == 2**64-1
True
read_unicodestring1(f)
>>> import io
>>> s = 'abcd\uabcd'
>>> enc = s.encode('utf-8')
>>> enc
b'abcd\xea\xaf\x8d'
>>> n = bytes([len(enc)]) # little-endian 1-byte length
>>> t = read_unicodestring1(io.BytesIO(n + enc + b'junk'))
>>> s == t
True
>>> read_unicodestring1(io.BytesIO(n + enc[:-1]))
Traceback (most recent call last):
...
ValueError: expected 7 bytes in a unicodestring1, but only 6 remain
read_unicodestring4(f)
>>> import io
>>> s = 'abcd\uabcd'
>>> enc = s.encode('utf-8')
>>> enc
b'abcd\xea\xaf\x8d'
>>> n = bytes([len(enc), 0, 0, 0]) # little-endian 4-byte length
>>> t = read_unicodestring4(io.BytesIO(n + enc + b'junk'))
>>> s == t
True
>>> read_unicodestring4(io.BytesIO(n + enc[:-1]))
Traceback (most recent call last):
...
ValueError: expected 7 bytes in a unicodestring4, but only 6 remain
read_unicodestring8(f)
>>> import io
>>> s = 'abcd\uabcd'
>>> enc = s.encode('utf-8')
>>> enc
b'abcd\xea\xaf\x8d'
>>> n = bytes([len(enc)]) + b'\0' * 7 # little-endian 8-byte length
>>> t = read_unicodestring8(io.BytesIO(n + enc + b'junk'))
>>> s == t
True
>>> read_unicodestring8(io.BytesIO(n + enc[:-1]))
Traceback (most recent call last):
...
ValueError: expected 7 bytes in a unicodestring8, but only 6 remain
read_unicodestringnl(f)
>>> import io
>>> read_unicodestringnl(io.BytesIO(b"abc\\uabcd\njunk")) == 'abc\uabcd'
True
TAKEN_FROM_ARGUMENT1 = -2
TAKEN_FROM_ARGUMENT4 = -3
TAKEN_FROM_ARGUMENT4U = -4
TAKEN_FROM_ARGUMENT8U = -5
UP_TO_NEWLINE = -1
anyobject = any
bytearray8 = <pickletools.ArgumentDescriptor object at 0x7f056605e4c0>
bytes1 = <pickletools.ArgumentDescriptor object at 0x7f056605e400>
bytes4 = <pickletools.ArgumentDescriptor object at 0x7f056605e440>
bytes8 = <pickletools.ArgumentDescriptor object at 0x7f056605e480>
bytes_types = (<class 'bytes'>, <class 'bytearray'>)
code2op = {'I': <pickletools.OpcodeInfo object at 0x7f0566061040>, 'J': <pickletools.OpcodeInfo object at 0x7f0566061100>, 'K': <pickletools.OpcodeInfo object at 0x7f0566061160>, 'M': <pickletools.OpcodeInfo object at 0x7f05660611c0>, 'L': <pickletools.OpcodeInfo object at 0x7f0566061220>, '\x8a': <pickletools.OpcodeInfo object at 0x7f0566061280>, '\x8b': <pickletools.OpcodeInfo object at 0x7f05660612e0>, 'S': <pickletools.OpcodeInfo object at 0x7f0566061340>, 'T': <pickletools.OpcodeInfo object at 0x7f05660613a0>, 'U': <pickletools.OpcodeInfo object at 0x7f0566061400>, 'B': <pickletools.OpcodeInfo object at 0x7f0566061460>, 'C': <pickletools.OpcodeInfo object at 0x7f05660614c0>, '\x8e': <pickletools.OpcodeInfo object at 0x7f0566061520>, '\x96': <pickletools.OpcodeInfo object at 0x7f0566061580>, '\x97': <pickletools.OpcodeInfo object at 0x7f05660615e0>, '\x98': <pickletools.OpcodeInfo object at 0x7f0566061640>, 'N': <pickletools.OpcodeInfo object at 0x7f05660616a0>, '\x88': <pickletools.OpcodeInfo object at 0x7f0566061700>, '\x89': <pickletools.OpcodeInfo object at 0x7f0566061760>, 'V': <pickletools.OpcodeInfo object at 0x7f05660617c0>, '\x8c': <pickletools.OpcodeInfo object at 0x7f0566061820>, 'X': <pickletools.OpcodeInfo object at 0x7f0566061880>, '\x8d': <pickletools.OpcodeInfo object at 0x7f05660618e0>, 'F': <pickletools.OpcodeInfo object at 0x7f0566061940>, 'G': <pickletools.OpcodeInfo object at 0x7f05660619a0>, ']': <pickletools.OpcodeInfo object at 0x7f0566061a00>, 'a': <pickletools.OpcodeInfo object at 0x7f0566061a60>, 'e': <pickletools.OpcodeInfo object at 0x7f0566061ac0>, 'l': <pickletools.OpcodeInfo object at 0x7f0566061b20>, ')': <pickletools.OpcodeInfo object at 0x7f0566061b80>, 't': <pickletools.OpcodeInfo object at 0x7f0566061be0>, '\x85': <pickletools.OpcodeInfo object at 0x7f0566061c40>, '\x86': <pickletools.OpcodeInfo object at 0x7f0566061ca0>, '\x87': <pickletools.OpcodeInfo object at 0x7f0566061d00>, '}': <pickletools.OpcodeInfo object at 0x7f0566061d60>, 'd': <pickletools.OpcodeInfo object at 0x7f0566061dc0>, 's': <pickletools.OpcodeInfo object at 0x7f0566061e20>, 'u': <pickletools.OpcodeInfo object at 0x7f0566061e80>, '\x8f': <pickletools.OpcodeInfo object at 0x7f0566061ee0>, '\x90': <pickletools.OpcodeInfo object at 0x7f0566061f40>, '\x91': <pickletools.OpcodeInfo object at 0x7f0566061fa0>, '0': <pickletools.OpcodeInfo object at 0x7f0566064040>, '2': <pickletools.OpcodeInfo object at 0x7f05660640a0>, '(': <pickletools.OpcodeInfo object at 0x7f0566064100>, '1': <pickletools.OpcodeInfo object at 0x7f0566064160>, 'g': <pickletools.OpcodeInfo object at 0x7f05660641c0>, 'h': <pickletools.OpcodeInfo object at 0x7f0566064220>, 'j': <pickletools.OpcodeInfo object at 0x7f0566064280>, 'p': <pickletools.OpcodeInfo object at 0x7f05660642e0>, 'q': <pickletools.OpcodeInfo object at 0x7f0566064340>, 'r': <pickletools.OpcodeInfo object at 0x7f05660643a0>, '\x94': <pickletools.OpcodeInfo object at 0x7f0566064400>, '\x82': <pickletools.OpcodeInfo object at 0x7f0566064460>, '\x83': <pickletools.OpcodeInfo object at 0x7f05660644c0>, '\x84': <pickletools.OpcodeInfo object at 0x7f0566064520>, 'c': <pickletools.OpcodeInfo object at 0x7f0566064580>, '\x93': <pickletools.OpcodeInfo object at 0x7f05660645e0>, 'R': <pickletools.OpcodeInfo object at 0x7f0566064640>, 'b': <pickletools.OpcodeInfo object at 0x7f05660646a0>, 'i': <pickletools.OpcodeInfo object at 0x7f0566064700>, 'o': <pickletools.OpcodeInfo object at 0x7f0566064760>, '\x81': <pickletools.OpcodeInfo object at 0x7f05660647c0>, '\x92': <pickletools.OpcodeInfo object at 0x7f0566064820>, '\x80': <pickletools.OpcodeInfo object at 0x7f0566064880>, '.': <pickletools.OpcodeInfo object at 0x7f05660648e0>, '\x95': <pickletools.OpcodeInfo object at 0x7f0566064940>, 'P': <pickletools.OpcodeInfo object at 0x7f05660649a0>, 'Q': <pickletools.OpcodeInfo object at 0x7f0566064a00>}
decimalnl_long = <pickletools.ArgumentDescriptor object at 0x7f056605e640>
decimalnl_short = <pickletools.ArgumentDescriptor object at 0x7f056605e600>
float8 = <pickletools.ArgumentDescriptor object at 0x7f056605e6c0>
floatnl = <pickletools.ArgumentDescriptor object at 0x7f056605e680>
int4 = <pickletools.ArgumentDescriptor object at 0x7f056605e200>
long1 = <pickletools.ArgumentDescriptor object at 0x7f056605e700>
long4 = <pickletools.ArgumentDescriptor object at 0x7f056605e740>
markobject = mark
opcodes = [<pickletools.OpcodeInfo object at 0x7f0566061040>, <pickletools.OpcodeInfo object at 0x7f0566061100>, <pickletools.OpcodeInfo object at 0x7f0566061160>, <pickletools.OpcodeInfo object at 0x7f05660611c0>, <pickletools.OpcodeInfo object at 0x7f0566061220>, <pickletools.OpcodeInfo object at 0x7f0566061280>, <pickletools.OpcodeInfo object at 0x7f05660612e0>, <pickletools.OpcodeInfo object at 0x7f0566061340>, <pickletools.OpcodeInfo object at 0x7f05660613a0>, <pickletools.OpcodeInfo object at 0x7f0566061400>, <pickletools.OpcodeInfo object at 0x7f0566061460>, <pickletools.OpcodeInfo object at 0x7f05660614c0>, <pickletools.OpcodeInfo object at 0x7f0566061520>, <pickletools.OpcodeInfo object at 0x7f0566061580>, <pickletools.OpcodeInfo object at 0x7f05660615e0>, <pickletools.OpcodeInfo object at 0x7f0566061640>, <pickletools.OpcodeInfo object at 0x7f05660616a0>, <pickletools.OpcodeInfo object at 0x7f0566061700>, <pickletools.OpcodeInfo object at 0x7f0566061760>, <pickletools.OpcodeInfo object at 0x7f05660617c0>, <pickletools.OpcodeInfo object at 0x7f0566061820>, <pickletools.OpcodeInfo object at 0x7f0566061880>, <pickletools.OpcodeInfo object at 0x7f05660618e0>, <pickletools.OpcodeInfo object at 0x7f0566061940>, <pickletools.OpcodeInfo object at 0x7f05660619a0>, <pickletools.OpcodeInfo object at 0x7f0566061a00>, <pickletools.OpcodeInfo object at 0x7f0566061a60>, <pickletools.OpcodeInfo object at 0x7f0566061ac0>, <pickletools.OpcodeInfo object at 0x7f0566061b20>, <pickletools.OpcodeInfo object at 0x7f0566061b80>, <pickletools.OpcodeInfo object at 0x7f0566061be0>, <pickletools.OpcodeInfo object at 0x7f0566061c40>, <pickletools.OpcodeInfo object at 0x7f0566061ca0>, <pickletools.OpcodeInfo object at 0x7f0566061d00>, <pickletools.OpcodeInfo object at 0x7f0566061d60>, <pickletools.OpcodeInfo object at 0x7f0566061dc0>, <pickletools.OpcodeInfo object at 0x7f0566061e20>, <pickletools.OpcodeInfo object at 0x7f0566061e80>, <pickletools.OpcodeInfo object at 0x7f0566061ee0>, <pickletools.OpcodeInfo object at 0x7f0566061f40>, <pickletools.OpcodeInfo object at 0x7f0566061fa0>, <pickletools.OpcodeInfo object at 0x7f0566064040>, <pickletools.OpcodeInfo object at 0x7f05660640a0>, <pickletools.OpcodeInfo object at 0x7f0566064100>, <pickletools.OpcodeInfo object at 0x7f0566064160>, <pickletools.OpcodeInfo object at 0x7f05660641c0>, <pickletools.OpcodeInfo object at 0x7f0566064220>, <pickletools.OpcodeInfo object at 0x7f0566064280>, <pickletools.OpcodeInfo object at 0x7f05660642e0>, <pickletools.OpcodeInfo object at 0x7f0566064340>, <pickletools.OpcodeInfo object at 0x7f05660643a0>, <pickletools.OpcodeInfo object at 0x7f0566064400>, <pickletools.OpcodeInfo object at 0x7f0566064460>, <pickletools.OpcodeInfo object at 0x7f05660644c0>, <pickletools.OpcodeInfo object at 0x7f0566064520>, <pickletools.OpcodeInfo object at 0x7f0566064580>, <pickletools.OpcodeInfo object at 0x7f05660645e0>, <pickletools.OpcodeInfo object at 0x7f0566064640>, <pickletools.OpcodeInfo object at 0x7f05660646a0>, <pickletools.OpcodeInfo object at 0x7f0566064700>, <pickletools.OpcodeInfo object at 0x7f0566064760>, <pickletools.OpcodeInfo object at 0x7f05660647c0>, <pickletools.OpcodeInfo object at 0x7f0566064820>, <pickletools.OpcodeInfo object at 0x7f0566064880>, <pickletools.OpcodeInfo object at 0x7f05660648e0>, <pickletools.OpcodeInfo object at 0x7f0566064940>, <pickletools.OpcodeInfo object at 0x7f05660649a0>, <pickletools.OpcodeInfo object at 0x7f0566064a00>]
pybool = bool
pybuffer = buffer
pybytearray = bytearray
pybytes = bytes
pybytes_or_str = bytes_or_str
pydict = dict
pyfloat = float
pyfrozenset = frozenset
pyint = int
pyinteger_or_bool = int_or_bool
pylist = list
pylong = int
pynone = None
pyset = set
pystring = bytes_or_str
pytuple = tuple
pyunicode = str
stackslice = stackslice
string1 = <pickletools.ArgumentDescriptor object at 0x7f056605e380>
string4 = <pickletools.ArgumentDescriptor object at 0x7f056605e3c0>
stringnl = <pickletools.ArgumentDescriptor object at 0x7f056605e2c0>
stringnl_noescape = <pickletools.ArgumentDescriptor object at 0x7f056605e300>
stringnl_noescape_pair = <pickletools.ArgumentDescriptor object at 0x7f056605e340>
uint1 = <pickletools.ArgumentDescriptor object at 0x7f056605e180>
uint2 = <pickletools.ArgumentDescriptor object at 0x7f056605e1c0>
uint4 = <pickletools.ArgumentDescriptor object at 0x7f056605e240>
uint8 = <pickletools.ArgumentDescriptor object at 0x7f056605e280>
unicodestring1 = <pickletools.ArgumentDescriptor object at 0x7f056605e540>
unicodestring4 = <pickletools.ArgumentDescriptor object at 0x7f056605e580>
unicodestring8 = <pickletools.ArgumentDescriptor object at 0x7f056605e5c0>
unicodestringnl = <pickletools.ArgumentDescriptor object at 0x7f056605e500>