Mercurial > cpython-lru_cache-weakref

annotate Lib/json/decoder.py @ 65989:458b05abf987

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Output served URL when running ssl_servers
author Antoine Pitrou <solipsis@pitrou.net>
date Fri, 05 Nov 2010 20:26:59 +0000
parents 6f4cf73032fc
children 3c9232a684ae
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
1 """Implementation of JSONDecoder
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
2 """
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
3 import binascii
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
4 import re
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
5 import sys
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
6 import struct
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
7
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
8 from json.scanner import make_scanner
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
9 try:
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
10 from _json import scanstring as c_scanstring
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
11 except ImportError:
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
12 c_scanstring = None
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
13
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
14 __all__ = ['JSONDecoder']
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
15
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
16 FLAGS = re.VERBOSE | re.MULTILINE | re.DOTALL
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
17
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
18 def _floatconstants():
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
19 _BYTES = binascii.unhexlify(b'7FF80000000000007FF0000000000000')
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
20 if sys.byteorder != 'big':
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
21 _BYTES = _BYTES[:8][::-1] + _BYTES[8:][::-1]
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
22 nan, inf = struct.unpack('dd', _BYTES)
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
23 return nan, inf, -inf
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
24
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
25 NaN, PosInf, NegInf = _floatconstants()
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
26
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
27
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
28 def linecol(doc, pos):
49524
00e467db6182 merge r66932 and add a few py3k only checks
Benjamin Peterson <benjamin@python.org>
parents: 48769
diff changeset
29 if isinstance(doc, bytes):
00e467db6182 merge r66932 and add a few py3k only checks
Benjamin Peterson <benjamin@python.org>
parents: 48769
diff changeset
30 newline = b'\n'
00e467db6182 merge r66932 and add a few py3k only checks
Benjamin Peterson <benjamin@python.org>
parents: 48769
diff changeset
31 else:
00e467db6182 merge r66932 and add a few py3k only checks
Benjamin Peterson <benjamin@python.org>
parents: 48769
diff changeset
32 newline = '\n'
00e467db6182 merge r66932 and add a few py3k only checks
Benjamin Peterson <benjamin@python.org>
parents: 48769
diff changeset
33 lineno = doc.count(newline, 0, pos) + 1
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
34 if lineno == 1:
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
35 colno = pos
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
36 else:
49524
00e467db6182 merge r66932 and add a few py3k only checks
Benjamin Peterson <benjamin@python.org>
parents: 48769
diff changeset
37 colno = pos - doc.rindex(newline, 0, pos)
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
38 return lineno, colno
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
39
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
40
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
41 def errmsg(msg, doc, pos, end=None):
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
42 # Note that this function is called from _json
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
43 lineno, colno = linecol(doc, pos)
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
44 if end is None:
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
45 fmt = '{0}: line {1} column {2} (char {3})'
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
46 return fmt.format(msg, lineno, colno, pos)
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
47 #fmt = '%s: line %d column %d (char %d)'
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
48 #return fmt % (msg, lineno, colno, pos)
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
49 endlineno, endcolno = linecol(doc, end)
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
50 fmt = '{0}: line {1} column {2} - line {3} column {4} (char {5} - {6})'
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
51 return fmt.format(msg, lineno, colno, endlineno, endcolno, pos, end)
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
52 #fmt = '%s: line %d column %d - line %d column %d (char %d - %d)'
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
53 #return fmt % (msg, lineno, colno, endlineno, endcolno, pos, end)
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
54
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
55
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
56 _CONSTANTS = {
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
57 '-Infinity': NegInf,
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
58 'Infinity': PosInf,
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
59 'NaN': NaN,
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
60 }
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
61
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
62
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
63 STRINGCHUNK = re.compile(r'(.*?)(["\\\x00-\x1f])', FLAGS)
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
64 BACKSLASH = {
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
65 '"': '"', '\\': '\\', '/': '/',
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
66 'b': '\b', 'f': '\f', 'n': '\n', 'r': '\r', 't': '\t',
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
67 }
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
68
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
69 def py_scanstring(s, end, strict=True,
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
70 _b=BACKSLASH, _m=STRINGCHUNK.match):
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
71 """Scan the string s for a JSON string. End is the index of the
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
72 character in s after the quote that started the JSON string.
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
73 Unescapes all valid JSON string escape sequences and raises ValueError
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
74 on attempt to decode an invalid string. If strict is False then literal
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
75 control characters are allowed in the string.
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
76
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
77 Returns a tuple of the decoded string and the index of the character in s
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
78 after the end quote."""
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
79 chunks = []
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
80 _append = chunks.append
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
81 begin = end - 1
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
82 while 1:
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
83 chunk = _m(s, end)
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
84 if chunk is None:
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
85 raise ValueError(
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
86 errmsg("Unterminated string starting at", s, begin))
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
87 end = chunk.end()
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
88 content, terminator = chunk.groups()
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
89 # Content is contains zero or more unescaped string characters
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
90 if content:
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
91 _append(content)
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
92 # Terminator is the end of string, a literal control character,
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
93 # or a backslash denoting that an escape sequence follows
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
94 if terminator == '"':
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
95 break
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
96 elif terminator != '\\':
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
97 if strict:
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
98 #msg = "Invalid control character %r at" % (terminator,)
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
99 msg = "Invalid control character {0!r} at".format(terminator)
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
100 raise ValueError(errmsg(msg, s, end))
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
101 else:
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
102 _append(terminator)
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
103 continue
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
104 try:
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
105 esc = s[end]
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
106 except IndexError:
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
107 raise ValueError(
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
108 errmsg("Unterminated string starting at", s, begin))
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
109 # If not a unicode escape sequence, must be in the lookup table
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
110 if esc != 'u':
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
111 try:
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
112 char = _b[esc]
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
113 except KeyError:
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
114 msg = "Invalid \\escape: {0!r}".format(esc)
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
115 raise ValueError(errmsg(msg, s, end))
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
116 end += 1
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
117 else:
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
118 esc = s[end + 1:end + 5]
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
119 next_end = end + 5
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
120 if len(esc) != 4:
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
121 msg = "Invalid \\uXXXX escape"
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
122 raise ValueError(errmsg(msg, s, end))
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
123 uni = int(esc, 16)
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
124 # Check for surrogate pair on UCS-4 systems
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
125 if 0xd800 <= uni <= 0xdbff and sys.maxunicode > 65535:
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
126 msg = "Invalid \\uXXXX\\uXXXX surrogate pair"
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
127 if not s[end + 5:end + 7] == '\\u':
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
128 raise ValueError(errmsg(msg, s, end))
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
129 esc2 = s[end + 7:end + 11]
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
130 if len(esc2) != 4:
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
131 raise ValueError(errmsg(msg, s, end))
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
132 uni2 = int(esc2, 16)
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
133 uni = 0x10000 + (((uni - 0xd800) << 10) | (uni2 - 0xdc00))
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
134 next_end += 6
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
135 char = chr(uni)
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
136
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
137 end = next_end
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
138 _append(char)
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
139 return ''.join(chunks), end
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
140
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
141
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
142 # Use speedup if available
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
143 scanstring = c_scanstring or py_scanstring
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
144
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
145 WHITESPACE = re.compile(r'[ \t\n\r]*', FLAGS)
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
146 WHITESPACE_STR = ' \t\n\r'
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
147
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
148
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
149 def JSONObject(s_and_end, strict, scan_once, object_hook, object_pairs_hook,
64443
c3ad883b940b Issue #7451: Improve decoding performance of JSON objects, and reduce
Antoine Pitrou <solipsis@pitrou.net>
parents: 63575
diff changeset
150 memo=None, _w=WHITESPACE.match, _ws=WHITESPACE_STR):
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
151 s, end = s_and_end
53296
93c929a2fac7 Forward port r70471: Add object_pairs_hook. Issue 5381.
Raymond Hettinger <python@rcn.com>
parents: 49524
diff changeset
152 pairs = []
93c929a2fac7 Forward port r70471: Add object_pairs_hook. Issue 5381.
Raymond Hettinger <python@rcn.com>
parents: 49524
diff changeset
153 pairs_append = pairs.append
64443
c3ad883b940b Issue #7451: Improve decoding performance of JSON objects, and reduce
Antoine Pitrou <solipsis@pitrou.net>
parents: 63575
diff changeset
154 # Backwards compatibility
c3ad883b940b Issue #7451: Improve decoding performance of JSON objects, and reduce
Antoine Pitrou <solipsis@pitrou.net>
parents: 63575
diff changeset
155 if memo is None:
c3ad883b940b Issue #7451: Improve decoding performance of JSON objects, and reduce
Antoine Pitrou <solipsis@pitrou.net>
parents: 63575
diff changeset
156 memo = {}
c3ad883b940b Issue #7451: Improve decoding performance of JSON objects, and reduce
Antoine Pitrou <solipsis@pitrou.net>
parents: 63575
diff changeset
157 memo_get = memo.setdefault
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
158 # Use a slice to prevent IndexError from being raised, the following
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
159 # check will raise a more specific ValueError if the string is empty
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
160 nextchar = s[end:end + 1]
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
161 # Normally we expect nextchar == '"'
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
162 if nextchar != '"':
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
163 if nextchar in _ws:
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
164 end = _w(s, end).end()
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
165 nextchar = s[end:end + 1]
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
166 # Trivial empty object
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
167 if nextchar == '}':
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
168 return pairs, end + 1
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
169 elif nextchar != '"':
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
170 raise ValueError(errmsg("Expecting property name", s, end))
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
171 end += 1
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
172 while True:
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
173 key, end = scanstring(s, end, strict)
64443
c3ad883b940b Issue #7451: Improve decoding performance of JSON objects, and reduce
Antoine Pitrou <solipsis@pitrou.net>
parents: 63575
diff changeset
174 key = memo_get(key, key)
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
175 # To skip some function call overhead we optimize the fast paths where
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
176 # the JSON key separator is ": " or just ":".
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
177 if s[end:end + 1] != ':':
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
178 end = _w(s, end).end()
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
179 if s[end:end + 1] != ':':
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
180 raise ValueError(errmsg("Expecting : delimiter", s, end))
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
181 end += 1
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
182
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
183 try:
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
184 if s[end] in _ws:
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
185 end += 1
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
186 if s[end] in _ws:
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
187 end = _w(s, end + 1).end()
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
188 except IndexError:
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
189 pass
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
190
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
191 try:
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
192 value, end = scan_once(s, end)
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
193 except StopIteration:
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
194 raise ValueError(errmsg("Expecting object", s, end))
53296
93c929a2fac7 Forward port r70471: Add object_pairs_hook. Issue 5381.
Raymond Hettinger <python@rcn.com>
parents: 49524
diff changeset
195 pairs_append((key, value))
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
196 try:
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
197 nextchar = s[end]
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
198 if nextchar in _ws:
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
199 end = _w(s, end + 1).end()
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
200 nextchar = s[end]
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
201 except IndexError:
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
202 nextchar = ''
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
203 end += 1
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
204
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
205 if nextchar == '}':
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
206 break
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
207 elif nextchar != ',':
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
208 raise ValueError(errmsg("Expecting , delimiter", s, end - 1))
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
209 end = _w(s, end).end()
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
210 nextchar = s[end:end + 1]
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
211 end += 1
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
212 if nextchar != '"':
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
213 raise ValueError(errmsg("Expecting property name", s, end - 1))
53296
93c929a2fac7 Forward port r70471: Add object_pairs_hook. Issue 5381.
Raymond Hettinger <python@rcn.com>
parents: 49524
diff changeset
214 if object_pairs_hook is not None:
93c929a2fac7 Forward port r70471: Add object_pairs_hook. Issue 5381.
Raymond Hettinger <python@rcn.com>
parents: 49524
diff changeset
215 result = object_pairs_hook(pairs)
93c929a2fac7 Forward port r70471: Add object_pairs_hook. Issue 5381.
Raymond Hettinger <python@rcn.com>
parents: 49524
diff changeset
216 return result, end
93c929a2fac7 Forward port r70471: Add object_pairs_hook. Issue 5381.
Raymond Hettinger <python@rcn.com>
parents: 49524
diff changeset
217 pairs = dict(pairs)
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
218 if object_hook is not None:
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
219 pairs = object_hook(pairs)
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
220 return pairs, end
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
221
64443
c3ad883b940b Issue #7451: Improve decoding performance of JSON objects, and reduce
Antoine Pitrou <solipsis@pitrou.net>
parents: 63575
diff changeset
222 def JSONArray(s_and_end, scan_once, _w=WHITESPACE.match, _ws=WHITESPACE_STR):
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
223 s, end = s_and_end
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
224 values = []
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
225 nextchar = s[end:end + 1]
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
226 if nextchar in _ws:
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
227 end = _w(s, end + 1).end()
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
228 nextchar = s[end:end + 1]
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
229 # Look-ahead for trivial empty array
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
230 if nextchar == ']':
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
231 return values, end + 1
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
232 _append = values.append
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
233 while True:
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
234 try:
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
235 value, end = scan_once(s, end)
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
236 except StopIteration:
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
237 raise ValueError(errmsg("Expecting object", s, end))
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
238 _append(value)
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
239 nextchar = s[end:end + 1]
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
240 if nextchar in _ws:
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
241 end = _w(s, end + 1).end()
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
242 nextchar = s[end:end + 1]
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
243 end += 1
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
244 if nextchar == ']':
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
245 break
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
246 elif nextchar != ',':
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
247 raise ValueError(errmsg("Expecting , delimiter", s, end))
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
248 try:
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
249 if s[end] in _ws:
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
250 end += 1
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
251 if s[end] in _ws:
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
252 end = _w(s, end + 1).end()
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
253 except IndexError:
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
254 pass
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
255
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
256 return values, end
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
257
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
258
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
259 class JSONDecoder(object):
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
260 """Simple JSON <http://json.org> decoder
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
261
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
262 Performs the following translations in decoding by default:
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
263
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
264 +---------------+-------------------+
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
265 | JSON | Python |
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
266 +===============+===================+
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
267 | object | dict |
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
268 +---------------+-------------------+
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
269 | array | list |
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
270 +---------------+-------------------+
63575
4874efd1f657 #9087: update json docstrings -- unicode and long do not exist anymore.
Georg Brandl <georg@python.org>
parents: 53667
diff changeset
271 | string | str |
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
272 +---------------+-------------------+
63575
4874efd1f657 #9087: update json docstrings -- unicode and long do not exist anymore.
Georg Brandl <georg@python.org>
parents: 53667
diff changeset
273 | number (int) | int |
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
274 +---------------+-------------------+
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
275 | number (real) | float |
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
276 +---------------+-------------------+
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
277 | true | True |
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
278 +---------------+-------------------+
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
279 | false | False |
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
280 +---------------+-------------------+
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
281 | null | None |
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
282 +---------------+-------------------+
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
283
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
284 It also understands ``NaN``, ``Infinity``, and ``-Infinity`` as
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
285 their corresponding ``float`` values, which is outside the JSON spec.
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
286
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
287 """
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
288
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
289 def __init__(self, object_hook=None, parse_float=None,
53296
93c929a2fac7 Forward port r70471: Add object_pairs_hook. Issue 5381.
Raymond Hettinger <python@rcn.com>
parents: 49524
diff changeset
290 parse_int=None, parse_constant=None, strict=True,
93c929a2fac7 Forward port r70471: Add object_pairs_hook. Issue 5381.
Raymond Hettinger <python@rcn.com>
parents: 49524
diff changeset
291 object_pairs_hook=None):
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
292 """``object_hook``, if specified, will be called with the result
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
293 of every JSON object decoded and its return value will be used in
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
294 place of the given ``dict``. This can be used to provide custom
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
295 deserializations (e.g. to support JSON-RPC class hinting).
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
296
65374
6f4cf73032fc #4785: document strict argument of JSONDecoder, plus add object_pairs_hook in the docstrings.
Georg Brandl <georg@python.org>
parents: 64443
diff changeset
297 ``object_pairs_hook``, if specified will be called with the result of
6f4cf73032fc #4785: document strict argument of JSONDecoder, plus add object_pairs_hook in the docstrings.
Georg Brandl <georg@python.org>
parents: 64443
diff changeset
298 every JSON object decoded with an ordered list of pairs. The return
6f4cf73032fc #4785: document strict argument of JSONDecoder, plus add object_pairs_hook in the docstrings.
Georg Brandl <georg@python.org>
parents: 64443
diff changeset
299 value of ``object_pairs_hook`` will be used instead of the ``dict``.
6f4cf73032fc #4785: document strict argument of JSONDecoder, plus add object_pairs_hook in the docstrings.
Georg Brandl <georg@python.org>
parents: 64443
diff changeset
300 This feature can be used to implement custom decoders that rely on the
6f4cf73032fc #4785: document strict argument of JSONDecoder, plus add object_pairs_hook in the docstrings.
Georg Brandl <georg@python.org>
parents: 64443
diff changeset
301 order that the key and value pairs are decoded (for example,
6f4cf73032fc #4785: document strict argument of JSONDecoder, plus add object_pairs_hook in the docstrings.
Georg Brandl <georg@python.org>
parents: 64443
diff changeset
302 collections.OrderedDict will remember the order of insertion). If
6f4cf73032fc #4785: document strict argument of JSONDecoder, plus add object_pairs_hook in the docstrings.
Georg Brandl <georg@python.org>
parents: 64443
diff changeset
303 ``object_hook`` is also defined, the ``object_pairs_hook`` takes
6f4cf73032fc #4785: document strict argument of JSONDecoder, plus add object_pairs_hook in the docstrings.
Georg Brandl <georg@python.org>
parents: 64443
diff changeset
304 priority.
6f4cf73032fc #4785: document strict argument of JSONDecoder, plus add object_pairs_hook in the docstrings.
Georg Brandl <georg@python.org>
parents: 64443
diff changeset
305
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
306 ``parse_float``, if specified, will be called with the string
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
307 of every JSON float to be decoded. By default this is equivalent to
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
308 float(num_str). This can be used to use another datatype or parser
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
309 for JSON floats (e.g. decimal.Decimal).
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
310
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
311 ``parse_int``, if specified, will be called with the string
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
312 of every JSON int to be decoded. By default this is equivalent to
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
313 int(num_str). This can be used to use another datatype or parser
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
314 for JSON integers (e.g. float).
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
315
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
316 ``parse_constant``, if specified, will be called with one of the
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
317 following strings: -Infinity, Infinity, NaN.
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
318 This can be used to raise an exception if invalid JSON numbers
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
319 are encountered.
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
320
65374
6f4cf73032fc #4785: document strict argument of JSONDecoder, plus add object_pairs_hook in the docstrings.
Georg Brandl <georg@python.org>
parents: 64443
diff changeset
321 If ``strict`` is false (true is the default), then control
6f4cf73032fc #4785: document strict argument of JSONDecoder, plus add object_pairs_hook in the docstrings.
Georg Brandl <georg@python.org>
parents: 64443
diff changeset
322 characters will be allowed inside strings. Control characters in
6f4cf73032fc #4785: document strict argument of JSONDecoder, plus add object_pairs_hook in the docstrings.
Georg Brandl <georg@python.org>
parents: 64443
diff changeset
323 this context are those with character codes in the 0-31 range,
6f4cf73032fc #4785: document strict argument of JSONDecoder, plus add object_pairs_hook in the docstrings.
Georg Brandl <georg@python.org>
parents: 64443
diff changeset
324 including ``'\\t'`` (tab), ``'\\n'``, ``'\\r'`` and ``'\\0'``.
6f4cf73032fc #4785: document strict argument of JSONDecoder, plus add object_pairs_hook in the docstrings.
Georg Brandl <georg@python.org>
parents: 64443
diff changeset
325
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
326 """
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
327 self.object_hook = object_hook
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
328 self.parse_float = parse_float or float
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
329 self.parse_int = parse_int or int
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
330 self.parse_constant = parse_constant or _CONSTANTS.__getitem__
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
331 self.strict = strict
53296
93c929a2fac7 Forward port r70471: Add object_pairs_hook. Issue 5381.
Raymond Hettinger <python@rcn.com>
parents: 49524
diff changeset
332 self.object_pairs_hook = object_pairs_hook
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
333 self.parse_object = JSONObject
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
334 self.parse_array = JSONArray
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
335 self.parse_string = scanstring
64443
c3ad883b940b Issue #7451: Improve decoding performance of JSON objects, and reduce
Antoine Pitrou <solipsis@pitrou.net>
parents: 63575
diff changeset
336 self.memo = {}
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
337 self.scan_once = make_scanner(self)
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
338
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
339
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
340 def decode(self, s, _w=WHITESPACE.match):
63575
4874efd1f657 #9087: update json docstrings -- unicode and long do not exist anymore.
Georg Brandl <georg@python.org>
parents: 53667
diff changeset
341 """Return the Python representation of ``s`` (a ``str`` instance
4874efd1f657 #9087: update json docstrings -- unicode and long do not exist anymore.
Georg Brandl <georg@python.org>
parents: 53667
diff changeset
342 containing a JSON document).
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
343
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
344 """
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
345 obj, end = self.raw_decode(s, idx=_w(s, 0).end())
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
346 end = _w(s, end).end()
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
347 if end != len(s):
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
348 raise ValueError(errmsg("Extra data", s, end, len(s)))
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
349 return obj
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
350
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
351 def raw_decode(self, s, idx=0):
63575
4874efd1f657 #9087: update json docstrings -- unicode and long do not exist anymore.
Georg Brandl <georg@python.org>
parents: 53667
diff changeset
352 """Decode a JSON document from ``s`` (a ``str`` beginning with
4874efd1f657 #9087: update json docstrings -- unicode and long do not exist anymore.
Georg Brandl <georg@python.org>
parents: 53667
diff changeset
353 a JSON document) and return a 2-tuple of the Python
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
354 representation and the index in ``s`` where the document ended.
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
355
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
356 This can be used to decode a JSON document from a string that may
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
357 have extraneous data at the end.
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
358
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
359 """
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
360 try:
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
361 obj, end = self.scan_once(s, idx)
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
362 except StopIteration:
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
363 raise ValueError("No JSON object could be decoded")
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
364 return obj, end