Mercurial > cpython-lru_cache-weakref

annotate Lib/json/decoder.py @ 81343:3a7cd8efc19f

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
#16852: merge with 3.3.
author Ezio Melotti <ezio.melotti@gmail.com>
date Thu, 10 Jan 2013 03:29:45 +0200
parents 17b4bf9e9f43
children 69f793cc34fc
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
1 """Implementation of JSONDecoder
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
2 """
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
3 import re
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
4
70085
c2853a54b29e #5723: Improve json tests to be executed with and without accelerations.
Ezio Melotti <ezio.melotti@gmail.com>
parents: 69319
diff changeset
5 from json import scanner
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
6 try:
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
7 from _json import scanstring as c_scanstring
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
8 except ImportError:
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
9 c_scanstring = None
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
10
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
11 __all__ = ['JSONDecoder']
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
12
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
13 FLAGS = re.VERBOSE | re.MULTILINE | re.DOTALL
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
14
80633
402bae9f8fe5 Cleanup json decoder: float() has builtin support of nan, +inf, -inf since Python 2.6
Victor Stinner <victor.stinner@gmail.com>
parents: 77841
diff changeset
15 NaN = float('nan')
402bae9f8fe5 Cleanup json decoder: float() has builtin support of nan, +inf, -inf since Python 2.6
Victor Stinner <victor.stinner@gmail.com>
parents: 77841
diff changeset
16 PosInf = float('inf')
402bae9f8fe5 Cleanup json decoder: float() has builtin support of nan, +inf, -inf since Python 2.6
Victor Stinner <victor.stinner@gmail.com>
parents: 77841
diff changeset
17 NegInf = float('-inf')
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
18
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
19
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
20 def linecol(doc, pos):
49524
00e467db6182 merge r66932 and add a few py3k only checks
Benjamin Peterson <benjamin@python.org>
parents: 48769
diff changeset
21 if isinstance(doc, bytes):
00e467db6182 merge r66932 and add a few py3k only checks
Benjamin Peterson <benjamin@python.org>
parents: 48769
diff changeset
22 newline = b'\n'
00e467db6182 merge r66932 and add a few py3k only checks
Benjamin Peterson <benjamin@python.org>
parents: 48769
diff changeset
23 else:
00e467db6182 merge r66932 and add a few py3k only checks
Benjamin Peterson <benjamin@python.org>
parents: 48769
diff changeset
24 newline = '\n'
00e467db6182 merge r66932 and add a few py3k only checks
Benjamin Peterson <benjamin@python.org>
parents: 48769
diff changeset
25 lineno = doc.count(newline, 0, pos) + 1
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
26 if lineno == 1:
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
27 colno = pos
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
28 else:
49524
00e467db6182 merge r66932 and add a few py3k only checks
Benjamin Peterson <benjamin@python.org>
parents: 48769
diff changeset
29 colno = pos - doc.rindex(newline, 0, pos)
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
30 return lineno, colno
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
31
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
32
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
33 def errmsg(msg, doc, pos, end=None):
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
34 # Note that this function is called from _json
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
35 lineno, colno = linecol(doc, pos)
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
36 if end is None:
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
37 fmt = '{0}: line {1} column {2} (char {3})'
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
38 return fmt.format(msg, lineno, colno, pos)
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
39 #fmt = '%s: line %d column %d (char %d)'
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
40 #return fmt % (msg, lineno, colno, pos)
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
41 endlineno, endcolno = linecol(doc, end)
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
42 fmt = '{0}: line {1} column {2} - line {3} column {4} (char {5} - {6})'
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
43 return fmt.format(msg, lineno, colno, endlineno, endcolno, pos, end)
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
44 #fmt = '%s: line %d column %d - line %d column %d (char %d - %d)'
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
45 #return fmt % (msg, lineno, colno, endlineno, endcolno, pos, end)
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
46
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
47
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
48 _CONSTANTS = {
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
49 '-Infinity': NegInf,
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
50 'Infinity': PosInf,
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
51 'NaN': NaN,
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
52 }
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
53
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
54
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
55 STRINGCHUNK = re.compile(r'(.*?)(["\\\x00-\x1f])', FLAGS)
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
56 BACKSLASH = {
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
57 '"': '"', '\\': '\\', '/': '/',
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
58 'b': '\b', 'f': '\f', 'n': '\n', 'r': '\r', 't': '\t',
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
59 }
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
60
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
61 def py_scanstring(s, end, strict=True,
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
62 _b=BACKSLASH, _m=STRINGCHUNK.match):
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
63 """Scan the string s for a JSON string. End is the index of the
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
64 character in s after the quote that started the JSON string.
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
65 Unescapes all valid JSON string escape sequences and raises ValueError
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
66 on attempt to decode an invalid string. If strict is False then literal
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
67 control characters are allowed in the string.
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
68
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
69 Returns a tuple of the decoded string and the index of the character in s
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
70 after the end quote."""
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
71 chunks = []
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
72 _append = chunks.append
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
73 begin = end - 1
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
74 while 1:
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
75 chunk = _m(s, end)
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
76 if chunk is None:
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
77 raise ValueError(
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
78 errmsg("Unterminated string starting at", s, begin))
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
79 end = chunk.end()
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
80 content, terminator = chunk.groups()
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
81 # Content is contains zero or more unescaped string characters
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
82 if content:
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
83 _append(content)
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
84 # Terminator is the end of string, a literal control character,
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
85 # or a backslash denoting that an escape sequence follows
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
86 if terminator == '"':
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
87 break
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
88 elif terminator != '\\':
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
89 if strict:
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
90 #msg = "Invalid control character %r at" % (terminator,)
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
91 msg = "Invalid control character {0!r} at".format(terminator)
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
92 raise ValueError(errmsg(msg, s, end))
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
93 else:
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
94 _append(terminator)
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
95 continue
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
96 try:
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
97 esc = s[end]
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
98 except IndexError:
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
99 raise ValueError(
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
100 errmsg("Unterminated string starting at", s, begin))
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
101 # If not a unicode escape sequence, must be in the lookup table
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
102 if esc != 'u':
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
103 try:
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
104 char = _b[esc]
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
105 except KeyError:
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
106 msg = "Invalid \\escape: {0!r}".format(esc)
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
107 raise ValueError(errmsg(msg, s, end))
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
108 end += 1
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
109 else:
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
110 esc = s[end + 1:end + 5]
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
111 next_end = end + 5
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
112 if len(esc) != 4:
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
113 msg = "Invalid \\uXXXX escape"
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
114 raise ValueError(errmsg(msg, s, end))
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
115 uni = int(esc, 16)
72475
8beaa9a37387 Implement PEP 393.
Martin v. Löwis <martin@v.loewis.de>
parents: 70090
diff changeset
116 if 0xd800 <= uni <= 0xdbff:
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
117 msg = "Invalid \\uXXXX\\uXXXX surrogate pair"
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
118 if not s[end + 5:end + 7] == '\\u':
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
119 raise ValueError(errmsg(msg, s, end))
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
120 esc2 = s[end + 7:end + 11]
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
121 if len(esc2) != 4:
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
122 raise ValueError(errmsg(msg, s, end))
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
123 uni2 = int(esc2, 16)
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
124 uni = 0x10000 + (((uni - 0xd800) << 10) | (uni2 - 0xdc00))
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
125 next_end += 6
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
126 char = chr(uni)
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
127
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
128 end = next_end
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
129 _append(char)
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
130 return ''.join(chunks), end
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
131
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
132
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
133 # Use speedup if available
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
134 scanstring = c_scanstring or py_scanstring
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
135
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
136 WHITESPACE = re.compile(r'[ \t\n\r]*', FLAGS)
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
137 WHITESPACE_STR = ' \t\n\r'
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
138
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
139
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
140 def JSONObject(s_and_end, strict, scan_once, object_hook, object_pairs_hook,
64443
c3ad883b940b Issue #7451: Improve decoding performance of JSON objects, and reduce
Antoine Pitrou <solipsis@pitrou.net>
parents: 63575
diff changeset
141 memo=None, _w=WHITESPACE.match, _ws=WHITESPACE_STR):
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
142 s, end = s_and_end
53296
93c929a2fac7 Forward port r70471: Add object_pairs_hook. Issue 5381.
Raymond Hettinger <python@rcn.com>
parents: 49524
diff changeset
143 pairs = []
93c929a2fac7 Forward port r70471: Add object_pairs_hook. Issue 5381.
Raymond Hettinger <python@rcn.com>
parents: 49524
diff changeset
144 pairs_append = pairs.append
64443
c3ad883b940b Issue #7451: Improve decoding performance of JSON objects, and reduce
Antoine Pitrou <solipsis@pitrou.net>
parents: 63575
diff changeset
145 # Backwards compatibility
c3ad883b940b Issue #7451: Improve decoding performance of JSON objects, and reduce
Antoine Pitrou <solipsis@pitrou.net>
parents: 63575
diff changeset
146 if memo is None:
c3ad883b940b Issue #7451: Improve decoding performance of JSON objects, and reduce
Antoine Pitrou <solipsis@pitrou.net>
parents: 63575
diff changeset
147 memo = {}
c3ad883b940b Issue #7451: Improve decoding performance of JSON objects, and reduce
Antoine Pitrou <solipsis@pitrou.net>
parents: 63575
diff changeset
148 memo_get = memo.setdefault
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
149 # Use a slice to prevent IndexError from being raised, the following
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
150 # check will raise a more specific ValueError if the string is empty
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
151 nextchar = s[end:end + 1]
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
152 # Normally we expect nextchar == '"'
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
153 if nextchar != '"':
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
154 if nextchar in _ws:
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
155 end = _w(s, end).end()
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
156 nextchar = s[end:end + 1]
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
157 # Trivial empty object
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
158 if nextchar == '}':
69319
a220458179ed #9233: Fix json.loads({}) to return a dict (instead of a list), when _json is not available.
Ezio Melotti
parents: 69317
diff changeset
159 if object_pairs_hook is not None:
a220458179ed #9233: Fix json.loads({}) to return a dict (instead of a list), when _json is not available.
Ezio Melotti
parents: 69317
diff changeset
160 result = object_pairs_hook(pairs)
a220458179ed #9233: Fix json.loads({}) to return a dict (instead of a list), when _json is not available.
Ezio Melotti
parents: 69317
diff changeset
161 return result, end
a220458179ed #9233: Fix json.loads({}) to return a dict (instead of a list), when _json is not available.
Ezio Melotti
parents: 69317
diff changeset
162 pairs = {}
a220458179ed #9233: Fix json.loads({}) to return a dict (instead of a list), when _json is not available.
Ezio Melotti
parents: 69317
diff changeset
163 if object_hook is not None:
a220458179ed #9233: Fix json.loads({}) to return a dict (instead of a list), when _json is not available.
Ezio Melotti
parents: 69317
diff changeset
164 pairs = object_hook(pairs)
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
165 return pairs, end + 1
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
166 elif nextchar != '"':
77840
9854520c8200 Issue #5067: improve some json error messages.
Antoine Pitrou <solipsis@pitrou.net>
parents: 70090
diff changeset
167 raise ValueError(errmsg(
9854520c8200 Issue #5067: improve some json error messages.
Antoine Pitrou <solipsis@pitrou.net>
parents: 70090
diff changeset
168 "Expecting property name enclosed in double quotes", s, end))
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
169 end += 1
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
170 while True:
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
171 key, end = scanstring(s, end, strict)
64443
c3ad883b940b Issue #7451: Improve decoding performance of JSON objects, and reduce
Antoine Pitrou <solipsis@pitrou.net>
parents: 63575
diff changeset
172 key = memo_get(key, key)
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
173 # To skip some function call overhead we optimize the fast paths where
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
174 # the JSON key separator is ": " or just ":".
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
175 if s[end:end + 1] != ':':
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
176 end = _w(s, end).end()
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
177 if s[end:end + 1] != ':':
77840
9854520c8200 Issue #5067: improve some json error messages.
Antoine Pitrou <solipsis@pitrou.net>
parents: 70090
diff changeset
178 raise ValueError(errmsg("Expecting ':' delimiter", s, end))
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
179 end += 1
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
180
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
181 try:
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
182 if s[end] in _ws:
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
183 end += 1
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
184 if s[end] in _ws:
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
185 end = _w(s, end + 1).end()
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
186 except IndexError:
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
187 pass
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
188
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
189 try:
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
190 value, end = scan_once(s, end)
81270
17b4bf9e9f43 #16009: JSON error messages now provide more information. Patch by Serhiy Storchaka.
Ezio Melotti <ezio.melotti@gmail.com>
parents: 80633
diff changeset
191 except StopIteration as err:
17b4bf9e9f43 #16009: JSON error messages now provide more information. Patch by Serhiy Storchaka.
Ezio Melotti <ezio.melotti@gmail.com>
parents: 80633
diff changeset
192 raise ValueError(errmsg("Expecting value", s, err.value)) from None
53296
93c929a2fac7 Forward port r70471: Add object_pairs_hook. Issue 5381.
Raymond Hettinger <python@rcn.com>
parents: 49524
diff changeset
193 pairs_append((key, value))
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
194 try:
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
195 nextchar = s[end]
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
196 if nextchar in _ws:
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
197 end = _w(s, end + 1).end()
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
198 nextchar = s[end]
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
199 except IndexError:
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
200 nextchar = ''
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
201 end += 1
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
202
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
203 if nextchar == '}':
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
204 break
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
205 elif nextchar != ',':
77840
9854520c8200 Issue #5067: improve some json error messages.
Antoine Pitrou <solipsis@pitrou.net>
parents: 70090
diff changeset
206 raise ValueError(errmsg("Expecting ',' delimiter", s, end - 1))
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
207 end = _w(s, end).end()
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
208 nextchar = s[end:end + 1]
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
209 end += 1
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
210 if nextchar != '"':
77840
9854520c8200 Issue #5067: improve some json error messages.
Antoine Pitrou <solipsis@pitrou.net>
parents: 70090
diff changeset
211 raise ValueError(errmsg(
9854520c8200 Issue #5067: improve some json error messages.
Antoine Pitrou <solipsis@pitrou.net>
parents: 70090
diff changeset
212 "Expecting property name enclosed in double quotes", s, end - 1))
53296
93c929a2fac7 Forward port r70471: Add object_pairs_hook. Issue 5381.
Raymond Hettinger <python@rcn.com>
parents: 49524
diff changeset
213 if object_pairs_hook is not None:
93c929a2fac7 Forward port r70471: Add object_pairs_hook. Issue 5381.
Raymond Hettinger <python@rcn.com>
parents: 49524
diff changeset
214 result = object_pairs_hook(pairs)
93c929a2fac7 Forward port r70471: Add object_pairs_hook. Issue 5381.
Raymond Hettinger <python@rcn.com>
parents: 49524
diff changeset
215 return result, end
93c929a2fac7 Forward port r70471: Add object_pairs_hook. Issue 5381.
Raymond Hettinger <python@rcn.com>
parents: 49524
diff changeset
216 pairs = dict(pairs)
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
217 if object_hook is not None:
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
218 pairs = object_hook(pairs)
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
219 return pairs, end
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
220
69317
7019fc1a9663 #9233: Fix json to work properly even when _json is not available.
Ezio Melotti
parents: 66470
diff changeset
221 def JSONArray(s_and_end, scan_once, _w=WHITESPACE.match, _ws=WHITESPACE_STR):
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
222 s, end = s_and_end
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
223 values = []
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
224 nextchar = s[end:end + 1]
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
225 if nextchar in _ws:
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
226 end = _w(s, end + 1).end()
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
227 nextchar = s[end:end + 1]
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
228 # Look-ahead for trivial empty array
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
229 if nextchar == ']':
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
230 return values, end + 1
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
231 _append = values.append
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
232 while True:
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
233 try:
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
234 value, end = scan_once(s, end)
81270
17b4bf9e9f43 #16009: JSON error messages now provide more information. Patch by Serhiy Storchaka.
Ezio Melotti <ezio.melotti@gmail.com>
parents: 80633
diff changeset
235 except StopIteration as err:
17b4bf9e9f43 #16009: JSON error messages now provide more information. Patch by Serhiy Storchaka.
Ezio Melotti <ezio.melotti@gmail.com>
parents: 80633
diff changeset
236 raise ValueError(errmsg("Expecting value", s, err.value)) from None
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
237 _append(value)
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
238 nextchar = s[end:end + 1]
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
239 if nextchar in _ws:
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
240 end = _w(s, end + 1).end()
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
241 nextchar = s[end:end + 1]
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
242 end += 1
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
243 if nextchar == ']':
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
244 break
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
245 elif nextchar != ',':
81270
17b4bf9e9f43 #16009: JSON error messages now provide more information. Patch by Serhiy Storchaka.
Ezio Melotti <ezio.melotti@gmail.com>
parents: 80633
diff changeset
246 raise ValueError(errmsg("Expecting ',' delimiter", s, end - 1))
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
247 try:
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
248 if s[end] in _ws:
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
249 end += 1
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
250 if s[end] in _ws:
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
251 end = _w(s, end + 1).end()
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
252 except IndexError:
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
253 pass
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
254
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
255 return values, end
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
256
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
257
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
258 class JSONDecoder(object):
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
259 """Simple JSON <http://json.org> decoder
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
260
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
261 Performs the following translations in decoding by default:
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
262
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
263 +---------------+-------------------+
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
264 | JSON | Python |
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
265 +===============+===================+
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
266 | object | dict |
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
267 +---------------+-------------------+
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
268 | array | list |
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
269 +---------------+-------------------+
63575
4874efd1f657 #9087: update json docstrings -- unicode and long do not exist anymore.
Georg Brandl <georg@python.org>
parents: 53667
diff changeset
270 | string | str |
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
271 +---------------+-------------------+
63575
4874efd1f657 #9087: update json docstrings -- unicode and long do not exist anymore.
Georg Brandl <georg@python.org>
parents: 53667
diff changeset
272 | number (int) | int |
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
273 +---------------+-------------------+
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
274 | number (real) | float |
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
275 +---------------+-------------------+
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
276 | true | True |
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
277 +---------------+-------------------+
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
278 | false | False |
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
279 +---------------+-------------------+
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
280 | null | None |
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
281 +---------------+-------------------+
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
282
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
283 It also understands ``NaN``, ``Infinity``, and ``-Infinity`` as
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
284 their corresponding ``float`` values, which is outside the JSON spec.
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
285
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
286 """
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
287
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
288 def __init__(self, object_hook=None, parse_float=None,
53296
93c929a2fac7 Forward port r70471: Add object_pairs_hook. Issue 5381.
Raymond Hettinger <python@rcn.com>
parents: 49524
diff changeset
289 parse_int=None, parse_constant=None, strict=True,
93c929a2fac7 Forward port r70471: Add object_pairs_hook. Issue 5381.
Raymond Hettinger <python@rcn.com>
parents: 49524
diff changeset
290 object_pairs_hook=None):
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
291 """``object_hook``, if specified, will be called with the result
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
292 of every JSON object decoded and its return value will be used in
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
293 place of the given ``dict``. This can be used to provide custom
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
294 deserializations (e.g. to support JSON-RPC class hinting).
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
295
66470
45e49269e0d8 Merged revisions 85530,85532-85534,85538-85543,85546-85548 via svnmerge from
Georg Brandl <georg@python.org>
parents: 63575
diff changeset
296 ``object_pairs_hook``, if specified will be called with the result of
45e49269e0d8 Merged revisions 85530,85532-85534,85538-85543,85546-85548 via svnmerge from
Georg Brandl <georg@python.org>
parents: 63575
diff changeset
297 every JSON object decoded with an ordered list of pairs. The return
45e49269e0d8 Merged revisions 85530,85532-85534,85538-85543,85546-85548 via svnmerge from
Georg Brandl <georg@python.org>
parents: 63575
diff changeset
298 value of ``object_pairs_hook`` will be used instead of the ``dict``.
45e49269e0d8 Merged revisions 85530,85532-85534,85538-85543,85546-85548 via svnmerge from
Georg Brandl <georg@python.org>
parents: 63575
diff changeset
299 This feature can be used to implement custom decoders that rely on the
45e49269e0d8 Merged revisions 85530,85532-85534,85538-85543,85546-85548 via svnmerge from
Georg Brandl <georg@python.org>
parents: 63575
diff changeset
300 order that the key and value pairs are decoded (for example,
45e49269e0d8 Merged revisions 85530,85532-85534,85538-85543,85546-85548 via svnmerge from
Georg Brandl <georg@python.org>
parents: 63575
diff changeset
301 collections.OrderedDict will remember the order of insertion). If
45e49269e0d8 Merged revisions 85530,85532-85534,85538-85543,85546-85548 via svnmerge from
Georg Brandl <georg@python.org>
parents: 63575
diff changeset
302 ``object_hook`` is also defined, the ``object_pairs_hook`` takes
45e49269e0d8 Merged revisions 85530,85532-85534,85538-85543,85546-85548 via svnmerge from
Georg Brandl <georg@python.org>
parents: 63575
diff changeset
303 priority.
45e49269e0d8 Merged revisions 85530,85532-85534,85538-85543,85546-85548 via svnmerge from
Georg Brandl <georg@python.org>
parents: 63575
diff changeset
304
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
305 ``parse_float``, if specified, will be called with the string
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
306 of every JSON float to be decoded. By default this is equivalent to
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
307 float(num_str). This can be used to use another datatype or parser
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
308 for JSON floats (e.g. decimal.Decimal).
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
309
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
310 ``parse_int``, if specified, will be called with the string
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
311 of every JSON int to be decoded. By default this is equivalent to
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
312 int(num_str). This can be used to use another datatype or parser
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
313 for JSON integers (e.g. float).
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
314
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
315 ``parse_constant``, if specified, will be called with one of the
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
316 following strings: -Infinity, Infinity, NaN.
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
317 This can be used to raise an exception if invalid JSON numbers
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
318 are encountered.
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
319
66470
45e49269e0d8 Merged revisions 85530,85532-85534,85538-85543,85546-85548 via svnmerge from
Georg Brandl <georg@python.org>
parents: 63575
diff changeset
320 If ``strict`` is false (true is the default), then control
45e49269e0d8 Merged revisions 85530,85532-85534,85538-85543,85546-85548 via svnmerge from
Georg Brandl <georg@python.org>
parents: 63575
diff changeset
321 characters will be allowed inside strings. Control characters in
45e49269e0d8 Merged revisions 85530,85532-85534,85538-85543,85546-85548 via svnmerge from
Georg Brandl <georg@python.org>
parents: 63575
diff changeset
322 this context are those with character codes in the 0-31 range,
45e49269e0d8 Merged revisions 85530,85532-85534,85538-85543,85546-85548 via svnmerge from
Georg Brandl <georg@python.org>
parents: 63575
diff changeset
323 including ``'\\t'`` (tab), ``'\\n'``, ``'\\r'`` and ``'\\0'``.
45e49269e0d8 Merged revisions 85530,85532-85534,85538-85543,85546-85548 via svnmerge from
Georg Brandl <georg@python.org>
parents: 63575
diff changeset
324
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
325 """
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
326 self.object_hook = object_hook
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
327 self.parse_float = parse_float or float
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
328 self.parse_int = parse_int or int
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
329 self.parse_constant = parse_constant or _CONSTANTS.__getitem__
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
330 self.strict = strict
53296
93c929a2fac7 Forward port r70471: Add object_pairs_hook. Issue 5381.
Raymond Hettinger <python@rcn.com>
parents: 49524
diff changeset
331 self.object_pairs_hook = object_pairs_hook
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
332 self.parse_object = JSONObject
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
333 self.parse_array = JSONArray
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
334 self.parse_string = scanstring
64443
c3ad883b940b Issue #7451: Improve decoding performance of JSON objects, and reduce
Antoine Pitrou <solipsis@pitrou.net>
parents: 63575
diff changeset
335 self.memo = {}
70085
c2853a54b29e #5723: Improve json tests to be executed with and without accelerations.
Ezio Melotti <ezio.melotti@gmail.com>
parents: 69319
diff changeset
336 self.scan_once = scanner.make_scanner(self)
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
337
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
338
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
339 def decode(self, s, _w=WHITESPACE.match):
63575
4874efd1f657 #9087: update json docstrings -- unicode and long do not exist anymore.
Georg Brandl <georg@python.org>
parents: 53667
diff changeset
340 """Return the Python representation of ``s`` (a ``str`` instance
4874efd1f657 #9087: update json docstrings -- unicode and long do not exist anymore.
Georg Brandl <georg@python.org>
parents: 53667
diff changeset
341 containing a JSON document).
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
342
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
343 """
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
344 obj, end = self.raw_decode(s, idx=_w(s, 0).end())
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
345 end = _w(s, end).end()
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
346 if end != len(s):
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
347 raise ValueError(errmsg("Extra data", s, end, len(s)))
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
348 return obj
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
349
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
350 def raw_decode(self, s, idx=0):
63575
4874efd1f657 #9087: update json docstrings -- unicode and long do not exist anymore.
Georg Brandl <georg@python.org>
parents: 53667
diff changeset
351 """Decode a JSON document from ``s`` (a ``str`` beginning with
4874efd1f657 #9087: update json docstrings -- unicode and long do not exist anymore.
Georg Brandl <georg@python.org>
parents: 53667
diff changeset
352 a JSON document) and return a 2-tuple of the Python
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
353 representation and the index in ``s`` where the document ended.
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
354
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
355 This can be used to decode a JSON document from a string that may
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
356 have extraneous data at the end.
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
357
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
358 """
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
359 try:
53667
f686aced02a3 port simplejson upgrade from the trunk #4136
Benjamin Peterson <benjamin@python.org>
parents: 53296
diff changeset
360 obj, end = self.scan_once(s, idx)
81270
17b4bf9e9f43 #16009: JSON error messages now provide more information. Patch by Serhiy Storchaka.
Ezio Melotti <ezio.melotti@gmail.com>
parents: 80633
diff changeset
361 except StopIteration as err:
17b4bf9e9f43 #16009: JSON error messages now provide more information. Patch by Serhiy Storchaka.
Ezio Melotti <ezio.melotti@gmail.com>
parents: 80633
diff changeset
362 raise ValueError(errmsg("Expecting value", s, err.value)) from None
46737
9f45979558a9 Merged revisions 62734,62736,62748,62769 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
diff changeset
363 return obj, end