Mercurial > cpython-lru_cache-weakref

annotate Lib/json/decoder.py @ 70626:aa07c1237f4e

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
(Merge 3.2) Issue #12016: Reindent decoders of HK and JP codecs
author Victor Stinner <victor.stinner@haypocalc.com>
date Fri, 03 Jun 2011 23:34:32 +0200
parents 18a192ae6db9
children 9854520c8200 8beaa9a37387
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
1 """Implementation of JSONDecoder
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
2 """
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
3 import binascii
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
4 import re
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
5 import sys
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
6 import struct
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
7
70085
c2853a54b29e #5723: Improve json tests to be executed with and without accelerations.
Ezio Melotti <ezio.melotti@gmail.com>
parents: 69319
diff changeset
8 from json import scanner
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
9 try:
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
10 from _json import scanstring as c_scanstring
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
11 except ImportError:
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
12 c_scanstring = None
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
13
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
14 __all__ = ['JSONDecoder']
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
15
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
16 FLAGS = re.VERBOSE | re.MULTILINE | re.DOTALL
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
17
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
18 def _floatconstants():
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
19 _BYTES = binascii.unhexlify(b'7FF80000000000007FF0000000000000')
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
20 if sys.byteorder != 'big':
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
21 _BYTES = _BYTES[:8][::-1] + _BYTES[8:][::-1]
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
22 nan, inf = struct.unpack('dd', _BYTES)
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
23 return nan, inf, -inf
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
24
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
25 NaN, PosInf, NegInf = _floatconstants()
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
26
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
27
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
28 def linecol(doc, pos):
49524
00e467db6182 merge r66932 and add a few py3k only checks
Benjamin Peterson <benjamin@python.org>
parents: 48769
diff changeset
29 if isinstance(doc, bytes):
00e467db6182 merge r66932 and add a few py3k only checks
Benjamin Peterson <benjamin@python.org>
parents: 48769
diff changeset
30 newline = b'\n'
00e467db6182 merge r66932 and add a few py3k only checks
Benjamin Peterson <benjamin@python.org>
parents: 48769
diff changeset
31 else:
00e467db6182 merge r66932 and add a few py3k only checks
Benjamin Peterson <benjamin@python.org>
parents: 48769
diff changeset
32 newline = '\n'
00e467db6182 merge r66932 and add a few py3k only checks
Benjamin Peterson <benjamin@python.org>
parents: 48769
diff changeset
33 lineno = doc.count(newline, 0, pos) + 1
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
34 if lineno == 1:
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
35 colno = pos
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
36 else:
49524
00e467db6182 merge r66932 and add a few py3k only checks
Benjamin Peterson <benjamin@python.org>
parents: 48769
diff changeset
37 colno = pos - doc.rindex(newline, 0, pos)
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
38 return lineno, colno
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
39
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
40
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
41 def errmsg(msg, doc, pos, end=None):
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
42 # Note that this function is called from _json
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
43 lineno, colno = linecol(doc, pos)
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
44 if end is None:
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
45 fmt = '{0}: line {1} column {2} (char {3})'
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
46 return fmt.format(msg, lineno, colno, pos)
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
47 #fmt = '%s: line %d column %d (char %d)'
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
48 #return fmt % (msg, lineno, colno, pos)
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
49 endlineno, endcolno = linecol(doc, end)
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
50 fmt = '{0}: line {1} column {2} - line {3} column {4} (char {5} - {6})'
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
51 return fmt.format(msg, lineno, colno, endlineno, endcolno, pos, end)
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
52 #fmt = '%s: line %d column %d - line %d column %d (char %d - %d)'
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
53 #return fmt % (msg, lineno, colno, endlineno, endcolno, pos, end)
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
54
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
55
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
56 _CONSTANTS = {
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
57 '-Infinity': NegInf,
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
58 'Infinity': PosInf,
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
59 'NaN': NaN,
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
60 }
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
61
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
62
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
63 STRINGCHUNK = re.compile(r'(.*?)(["\\\x00-\x1f])', FLAGS)
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
64 BACKSLASH = {
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
65 '"': '"', '\\': '\\', '/': '/',
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
66 'b': '\b', 'f': '\f', 'n': '\n', 'r': '\r', 't': '\t',
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
67 }
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
68
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
69 def py_scanstring(s, end, strict=True,
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
70 _b=BACKSLASH, _m=STRINGCHUNK.match):
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
71 """Scan the string s for a JSON string. End is the index of the
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
72 character in s after the quote that started the JSON string.
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
73 Unescapes all valid JSON string escape sequences and raises ValueError
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
74 on attempt to decode an invalid string. If strict is False then literal
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
75 control characters are allowed in the string.
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
76
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
77 Returns a tuple of the decoded string and the index of the character in s
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
78 after the end quote."""
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
79 chunks = []
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
80 _append = chunks.append
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
81 begin = end - 1
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
82 while 1:
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
83 chunk = _m(s, end)
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
84 if chunk is None:
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
85 raise ValueError(
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
86 errmsg("Unterminated string starting at", s, begin))
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
87 end = chunk.end()
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
88 content, terminator = chunk.groups()
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
89 # Content is contains zero or more unescaped string characters
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
90 if content:
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
91 _append(content)
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
92 # Terminator is the end of string, a literal control character,
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
93 # or a backslash denoting that an escape sequence follows
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
94 if terminator == '"':
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
95 break
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
96 elif terminator != '\\':
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
97 if strict:
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
98 #msg = "Invalid control character %r at" % (terminator,)
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
99 msg = "Invalid control character {0!r} at".format(terminator)
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
100 raise ValueError(errmsg(msg, s, end))
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
101 else:
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
102 _append(terminator)
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
103 continue
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
104 try:
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
105 esc = s[end]
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
106 except IndexError:
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
107 raise ValueError(
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
108 errmsg("Unterminated string starting at", s, begin))
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
109 # If not a unicode escape sequence, must be in the lookup table
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
110 if esc != 'u':
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
111 try:
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
112 char = _b[esc]
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
113 except KeyError:
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
114 msg = "Invalid \\escape: {0!r}".format(esc)
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
115 raise ValueError(errmsg(msg, s, end))
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
116 end += 1
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
117 else:
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
118 esc = s[end + 1:end + 5]
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
119 next_end = end + 5
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
120 if len(esc) != 4:
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
121 msg = "Invalid \\uXXXX escape"
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
122 raise ValueError(errmsg(msg, s, end))
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
123 uni = int(esc, 16)
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
124 # Check for surrogate pair on UCS-4 systems
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
125 if 0xd800 <= uni <= 0xdbff and sys.maxunicode > 65535:
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
126 msg = "Invalid \\uXXXX\\uXXXX surrogate pair"
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
127 if not s[end + 5:end + 7] == '\\u':
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
128 raise ValueError(errmsg(msg, s, end))
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
129 esc2 = s[end + 7:end + 11]
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
130 if len(esc2) != 4:
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
131 raise ValueError(errmsg(msg, s, end))
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
132 uni2 = int(esc2, 16)
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
133 uni = 0x10000 + (((uni - 0xd800) << 10) | (uni2 - 0xdc00))
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
134 next_end += 6
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
135 char = chr(uni)
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
136
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
137 end = next_end
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
138 _append(char)
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
139 return ''.join(chunks), end
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
140
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
141
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
142 # Use speedup if available
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
143 scanstring = c_scanstring or py_scanstring
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
144
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
145 WHITESPACE = re.compile(r'[ \t\n\r]*', FLAGS)
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
146 WHITESPACE_STR = ' \t\n\r'
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
147
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
148
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
149 def JSONObject(s_and_end, strict, scan_once, object_hook, object_pairs_hook,
64443
c3ad883b940b Issue #7451: Improve decoding performance of JSON objects, and reduce
Antoine Pitrou <solipsis@pitrou.net>
parents: 63575
diff changeset
150 memo=None, _w=WHITESPACE.match, _ws=WHITESPACE_STR):
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
151 s, end = s_and_end
53296
93c929a2fac7 Forward port r70471: Add object_pairs_hook. Issue 5381.
Raymond Hettinger <python@rcn.com>
parents: 49524
diff changeset
152 pairs = []
93c929a2fac7 Forward port r70471: Add object_pairs_hook. Issue 5381.
Raymond Hettinger <python@rcn.com>
parents: 49524
diff changeset
153 pairs_append = pairs.append
64443
c3ad883b940b Issue #7451: Improve decoding performance of JSON objects, and reduce
Antoine Pitrou <solipsis@pitrou.net>
parents: 63575
diff changeset
154 # Backwards compatibility
c3ad883b940b Issue #7451: Improve decoding performance of JSON objects, and reduce
Antoine Pitrou <solipsis@pitrou.net>
parents: 63575
diff changeset
155 if memo is None:
c3ad883b940b Issue #7451: Improve decoding performance of JSON objects, and reduce
Antoine Pitrou <solipsis@pitrou.net>
parents: 63575
diff changeset
156 memo = {}
c3ad883b940b Issue #7451: Improve decoding performance of JSON objects, and reduce
Antoine Pitrou <solipsis@pitrou.net>
parents: 63575
diff changeset
157 memo_get = memo.setdefault
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
158 # Use a slice to prevent IndexError from being raised, the following
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
159 # check will raise a more specific ValueError if the string is empty
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
160 nextchar = s[end:end + 1]
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
161 # Normally we expect nextchar == '"'
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
162 if nextchar != '"':
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
163 if nextchar in _ws:
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
164 end = _w(s, end).end()
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
165 nextchar = s[end:end + 1]
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
166 # Trivial empty object
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
167 if nextchar == '}':
69319
a220458179ed #9233: Fix json.loads({}) to return a dict (instead of a list), when _json is not available.
Ezio Melotti
parents: 69317
diff changeset
168 if object_pairs_hook is not None:
a220458179ed #9233: Fix json.loads({}) to return a dict (instead of a list), when _json is not available.
Ezio Melotti
parents: 69317
diff changeset
169 result = object_pairs_hook(pairs)
a220458179ed #9233: Fix json.loads({}) to return a dict (instead of a list), when _json is not available.
Ezio Melotti
parents: 69317
diff changeset
170 return result, end
a220458179ed #9233: Fix json.loads({}) to return a dict (instead of a list), when _json is not available.
Ezio Melotti
parents: 69317
diff changeset
171 pairs = {}
a220458179ed #9233: Fix json.loads({}) to return a dict (instead of a list), when _json is not available.
Ezio Melotti
parents: 69317
diff changeset
172 if object_hook is not None:
a220458179ed #9233: Fix json.loads({}) to return a dict (instead of a list), when _json is not available.
Ezio Melotti
parents: 69317
diff changeset
173 pairs = object_hook(pairs)
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
174 return pairs, end + 1
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
175 elif nextchar != '"':
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
176 raise ValueError(errmsg("Expecting property name", s, end))
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
177 end += 1
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
178 while True:
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
179 key, end = scanstring(s, end, strict)
64443
c3ad883b940b Issue #7451: Improve decoding performance of JSON objects, and reduce
Antoine Pitrou <solipsis@pitrou.net>
parents: 63575
diff changeset
180 key = memo_get(key, key)
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
181 # To skip some function call overhead we optimize the fast paths where
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
182 # the JSON key separator is ": " or just ":".
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
183 if s[end:end + 1] != ':':
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
184 end = _w(s, end).end()
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
185 if s[end:end + 1] != ':':
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
186 raise ValueError(errmsg("Expecting : delimiter", s, end))
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
187 end += 1
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
188
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
189 try:
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
190 if s[end] in _ws:
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
191 end += 1
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
192 if s[end] in _ws:
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
193 end = _w(s, end + 1).end()
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
194 except IndexError:
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
195 pass
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
196
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
197 try:
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
198 value, end = scan_once(s, end)
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
199 except StopIteration:
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
200 raise ValueError(errmsg("Expecting object", s, end))
53296
93c929a2fac7 Forward port r70471: Add object_pairs_hook. Issue 5381.
Raymond Hettinger <python@rcn.com>
parents: 49524
diff changeset
201 pairs_append((key, value))
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
202 try:
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
203 nextchar = s[end]
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
204 if nextchar in _ws:
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
205 end = _w(s, end + 1).end()
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
206 nextchar = s[end]
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
207 except IndexError:
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
208 nextchar = ''
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
209 end += 1
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
210
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
211 if nextchar == '}':
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
212 break
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
213 elif nextchar != ',':
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
214 raise ValueError(errmsg("Expecting , delimiter", s, end - 1))
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
215 end = _w(s, end).end()
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
216 nextchar = s[end:end + 1]
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
217 end += 1
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
218 if nextchar != '"':
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
219 raise ValueError(errmsg("Expecting property name", s, end - 1))
53296
93c929a2fac7 Forward port r70471: Add object_pairs_hook. Issue 5381.
Raymond Hettinger <python@rcn.com>
parents: 49524
diff changeset
220 if object_pairs_hook is not None:
93c929a2fac7 Forward port r70471: Add object_pairs_hook. Issue 5381.
Raymond Hettinger <python@rcn.com>
parents: 49524
diff changeset
221 result = object_pairs_hook(pairs)
93c929a2fac7 Forward port r70471: Add object_pairs_hook. Issue 5381.
Raymond Hettinger <python@rcn.com>
parents: 49524
diff changeset
222 return result, end
93c929a2fac7 Forward port r70471: Add object_pairs_hook. Issue 5381.
Raymond Hettinger <python@rcn.com>
parents: 49524
diff changeset
223 pairs = dict(pairs)
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
224 if object_hook is not None:
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
225 pairs = object_hook(pairs)
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
226 return pairs, end
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
227
69317
7019fc1a9663 #9233: Fix json to work properly even when _json is not available.
Ezio Melotti
parents: 66470
diff changeset
228 def JSONArray(s_and_end, scan_once, _w=WHITESPACE.match, _ws=WHITESPACE_STR):
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
229 s, end = s_and_end
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
230 values = []
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
231 nextchar = s[end:end + 1]
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
232 if nextchar in _ws:
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
233 end = _w(s, end + 1).end()
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
234 nextchar = s[end:end + 1]
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
235 # Look-ahead for trivial empty array
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
236 if nextchar == ']':
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
237 return values, end + 1
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
238 _append = values.append
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
239 while True:
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
240 try:
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
241 value, end = scan_once(s, end)
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
242 except StopIteration:
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
243 raise ValueError(errmsg("Expecting object", s, end))
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
244 _append(value)
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
245 nextchar = s[end:end + 1]
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
246 if nextchar in _ws:
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
247 end = _w(s, end + 1).end()
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
248 nextchar = s[end:end + 1]
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
249 end += 1
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
250 if nextchar == ']':
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
251 break
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
252 elif nextchar != ',':
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
253 raise ValueError(errmsg("Expecting , delimiter", s, end))
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
254 try:
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
255 if s[end] in _ws:
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
256 end += 1
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
257 if s[end] in _ws:
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
258 end = _w(s, end + 1).end()
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
259 except IndexError:
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
260 pass
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
261
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
262 return values, end
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
263
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
264
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
265 class JSONDecoder(object):
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
266 """Simple JSON <http://json.org> decoder
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
267
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
268 Performs the following translations in decoding by default:
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
269
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
270 +---------------+-------------------+
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
271 | JSON | Python |
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
272 +===============+===================+
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
273 | object | dict |
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
274 +---------------+-------------------+
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
275 | array | list |
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
276 +---------------+-------------------+
63575
4874efd1f657 #9087: update json docstrings -- unicode and long do not exist anymore.
Georg Brandl <georg@python.org>
parents: 53667
diff changeset
277 | string | str |
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
278 +---------------+-------------------+
63575
4874efd1f657 #9087: update json docstrings -- unicode and long do not exist anymore.
Georg Brandl <georg@python.org>
parents: 53667
diff changeset
279 | number (int) | int |
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
280 +---------------+-------------------+
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
281 | number (real) | float |
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
282 +---------------+-------------------+
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
283 | true | True |
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
284 +---------------+-------------------+
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
285 | false | False |
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
286 +---------------+-------------------+
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
287 | null | None |
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
288 +---------------+-------------------+
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
289
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
290 It also understands ``NaN``, ``Infinity``, and ``-Infinity`` as
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
291 their corresponding ``float`` values, which is outside the JSON spec.
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
292
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
293 """
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
294
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
295 def __init__(self, object_hook=None, parse_float=None,
53296
93c929a2fac7 Forward port r70471: Add object_pairs_hook. Issue 5381.
Raymond Hettinger <python@rcn.com>
parents: 49524
diff changeset
296 parse_int=None, parse_constant=None, strict=True,
93c929a2fac7 Forward port r70471: Add object_pairs_hook. Issue 5381.
Raymond Hettinger <python@rcn.com>
parents: 49524
diff changeset
297 object_pairs_hook=None):
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
298 """``object_hook``, if specified, will be called with the result
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
299 of every JSON object decoded and its return value will be used in
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
300 place of the given ``dict``. This can be used to provide custom
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
301 deserializations (e.g. to support JSON-RPC class hinting).
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
302
66470
45e49269e0d8 Merged revisions 85530,85532-85534,85538-85543,85546-85548 via svnmerge from
Georg Brandl <georg@python.org>
parents: 63575
diff changeset
303 ``object_pairs_hook``, if specified will be called with the result of
45e49269e0d8 Merged revisions 85530,85532-85534,85538-85543,85546-85548 via svnmerge from
Georg Brandl <georg@python.org>
parents: 63575
diff changeset
304 every JSON object decoded with an ordered list of pairs. The return
45e49269e0d8 Merged revisions 85530,85532-85534,85538-85543,85546-85548 via svnmerge from
Georg Brandl <georg@python.org>
parents: 63575
diff changeset
305 value of ``object_pairs_hook`` will be used instead of the ``dict``.
45e49269e0d8 Merged revisions 85530,85532-85534,85538-85543,85546-85548 via svnmerge from
Georg Brandl <georg@python.org>
parents: 63575
diff changeset
306 This feature can be used to implement custom decoders that rely on the
45e49269e0d8 Merged revisions 85530,85532-85534,85538-85543,85546-85548 via svnmerge from
Georg Brandl <georg@python.org>
parents: 63575
diff changeset
307 order that the key and value pairs are decoded (for example,
45e49269e0d8 Merged revisions 85530,85532-85534,85538-85543,85546-85548 via svnmerge from
Georg Brandl <georg@python.org>
parents: 63575
diff changeset
308 collections.OrderedDict will remember the order of insertion). If
45e49269e0d8 Merged revisions 85530,85532-85534,85538-85543,85546-85548 via svnmerge from
Georg Brandl <georg@python.org>
parents: 63575
diff changeset
309 ``object_hook`` is also defined, the ``object_pairs_hook`` takes
45e49269e0d8 Merged revisions 85530,85532-85534,85538-85543,85546-85548 via svnmerge from
Georg Brandl <georg@python.org>
parents: 63575
diff changeset
310 priority.
45e49269e0d8 Merged revisions 85530,85532-85534,85538-85543,85546-85548 via svnmerge from
Georg Brandl <georg@python.org>
parents: 63575
diff changeset
311
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
312 ``parse_float``, if specified, will be called with the string
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
313 of every JSON float to be decoded. By default this is equivalent to
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
314 float(num_str). This can be used to use another datatype or parser
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
315 for JSON floats (e.g. decimal.Decimal).
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
316
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
317 ``parse_int``, if specified, will be called with the string
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
318 of every JSON int to be decoded. By default this is equivalent to
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
319 int(num_str). This can be used to use another datatype or parser
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
320 for JSON integers (e.g. float).
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
321
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
322 ``parse_constant``, if specified, will be called with one of the
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
323 following strings: -Infinity, Infinity, NaN.
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
324 This can be used to raise an exception if invalid JSON numbers
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
325 are encountered.
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
326
66470
45e49269e0d8 Merged revisions 85530,85532-85534,85538-85543,85546-85548 via svnmerge from
Georg Brandl <georg@python.org>
parents: 63575
diff changeset
327 If ``strict`` is false (true is the default), then control
45e49269e0d8 Merged revisions 85530,85532-85534,85538-85543,85546-85548 via svnmerge from
Georg Brandl <georg@python.org>
parents: 63575
diff changeset
328 characters will be allowed inside strings. Control characters in
45e49269e0d8 Merged revisions 85530,85532-85534,85538-85543,85546-85548 via svnmerge from
Georg Brandl <georg@python.org>
parents: 63575
diff changeset
329 this context are those with character codes in the 0-31 range,
45e49269e0d8 Merged revisions 85530,85532-85534,85538-85543,85546-85548 via svnmerge from
Georg Brandl <georg@python.org>
parents: 63575
diff changeset
330 including ``'\\t'`` (tab), ``'\\n'``, ``'\\r'`` and ``'\\0'``.
45e49269e0d8 Merged revisions 85530,85532-85534,85538-85543,85546-85548 via svnmerge from
Georg Brandl <georg@python.org>
parents: 63575
diff changeset
331
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
332 """
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
333 self.object_hook = object_hook
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
334 self.parse_float = parse_float or float
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
335 self.parse_int = parse_int or int
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
336 self.parse_constant = parse_constant or _CONSTANTS.__getitem__
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
337 self.strict = strict
53296
93c929a2fac7 Forward port r70471: Add object_pairs_hook. Issue 5381.
Raymond Hettinger <python@rcn.com>
parents: 49524
diff changeset
338 self.object_pairs_hook = object_pairs_hook
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
339 self.parse_object = JSONObject
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
340 self.parse_array = JSONArray
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
341 self.parse_string = scanstring
64443
c3ad883b940b Issue #7451: Improve decoding performance of JSON objects, and reduce
Antoine Pitrou <solipsis@pitrou.net>
parents: 63575
diff changeset
342 self.memo = {}
70085
c2853a54b29e #5723: Improve json tests to be executed with and without accelerations.
Ezio Melotti <ezio.melotti@gmail.com>
parents: 69319
diff changeset
343 self.scan_once = scanner.make_scanner(self)
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
344
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
345
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
346 def decode(self, s, _w=WHITESPACE.match):
63575
4874efd1f657 #9087: update json docstrings -- unicode and long do not exist anymore.
Georg Brandl <georg@python.org>
parents: 53667
diff changeset
347 """Return the Python representation of ``s`` (a ``str`` instance
4874efd1f657 #9087: update json docstrings -- unicode and long do not exist anymore.
Georg Brandl <georg@python.org>
parents: 53667
diff changeset
348 containing a JSON document).
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
349
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
350 """
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
351 obj, end = self.raw_decode(s, idx=_w(s, 0).end())
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
352 end = _w(s, end).end()
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
353 if end != len(s):
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
354 raise ValueError(errmsg("Extra data", s, end, len(s)))
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
355 return obj
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
356
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
357 def raw_decode(self, s, idx=0):
63575
4874efd1f657 #9087: update json docstrings -- unicode and long do not exist anymore.
Georg Brandl <georg@python.org>
parents: 53667
diff changeset
358 """Decode a JSON document from ``s`` (a ``str`` beginning with
4874efd1f657 #9087: update json docstrings -- unicode and long do not exist anymore.
Georg Brandl <georg@python.org>
parents: 53667
diff changeset
359 a JSON document) and return a 2-tuple of the Python
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
360 representation and the index in ``s`` where the document ended.
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
361
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
362 This can be used to decode a JSON document from a string that may
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
363 have extraneous data at the end.
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
364
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
365 """
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
366 try:
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
367 obj, end = self.scan_once(s, idx)
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
368 except StopIteration:
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
369 raise ValueError("No JSON object could be decoded")
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
370 return obj, end