Mercurial > cpython-lru_cache-weakref

annotate Parser/node.c @ 24142:21bd86e00bce 2.2

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
PyNode_AddChild(): Backporting an aggressive over-allocation policy when a parse node grows a very large number of children. This sidesteps platform realloc() disasters on several platforms.
author Tim Peters <tim.peters@gmail.com>
date Mon, 08 Jul 2002 19:35:56 +0000
parents fb6b51033c3b
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
14391
1fd64b3af760 Move down the INT_MAX logic, because HAVE_LIMITS_H was always undefined
Vladimir Marangozov <vladimir.marangozov@t-online.de>
parents: 14375
diff changeset
1 /* Parse tree node implementation */
1fd64b3af760 Move down the INT_MAX logic, because HAVE_LIMITS_H was always undefined
Vladimir Marangozov <vladimir.marangozov@t-online.de>
parents: 14375
diff changeset
2
14871
fb6b51033c3b More limits.h stuff in node.c.
Tim Peters <tim.peters@gmail.com>
parents: 14391
diff changeset
3 #include "Python.h"
14391
1fd64b3af760 Move down the INT_MAX logic, because HAVE_LIMITS_H was always undefined
Vladimir Marangozov <vladimir.marangozov@t-online.de>
parents: 14375
diff changeset
4 #include "node.h"
1fd64b3af760 Move down the INT_MAX logic, because HAVE_LIMITS_H was always undefined
Vladimir Marangozov <vladimir.marangozov@t-online.de>
parents: 14375
diff changeset
5 #include "errcode.h"
145
853b736a6a9e Added copyright notice.
Guido van Rossum <guido@python.org>
parents: 86
diff changeset
6
6
fc6fcd7df4f7 Initial revision
Guido van Rossum <guido@python.org>
parents:
diff changeset
7 node *
13659
e78153e179fb Mass ANSIfication.
Thomas Wouters <thomas@python.org>
parents: 13447
diff changeset
8 PyNode_New(int type)
6
fc6fcd7df4f7 Initial revision
Guido van Rossum <guido@python.org>
parents:
diff changeset
9 {
5379
1c9a84e0981d Another directory quickly renamed.
Guido van Rossum <guido@python.org>
parents: 4503
diff changeset
10 node *n = PyMem_NEW(node, 1);
6
fc6fcd7df4f7 Initial revision
Guido van Rossum <guido@python.org>
parents:
diff changeset
11 if (n == NULL)
fc6fcd7df4f7 Initial revision
Guido van Rossum <guido@python.org>
parents:
diff changeset
12 return NULL;
fc6fcd7df4f7 Initial revision
Guido van Rossum <guido@python.org>
parents:
diff changeset
13 n->n_type = type;
fc6fcd7df4f7 Initial revision
Guido van Rossum <guido@python.org>
parents:
diff changeset
14 n->n_str = NULL;
86
a6934380c6e7 "Compiling" version
Guido van Rossum <guido@python.org>
parents: 76
diff changeset
15 n->n_lineno = 0;
6
fc6fcd7df4f7 Initial revision
Guido van Rossum <guido@python.org>
parents:
diff changeset
16 n->n_nchildren = 0;
fc6fcd7df4f7 Initial revision
Guido van Rossum <guido@python.org>
parents:
diff changeset
17 n->n_child = NULL;
fc6fcd7df4f7 Initial revision
Guido van Rossum <guido@python.org>
parents:
diff changeset
18 return n;
fc6fcd7df4f7 Initial revision
Guido van Rossum <guido@python.org>
parents:
diff changeset
19 }
fc6fcd7df4f7 Initial revision
Guido van Rossum <guido@python.org>
parents:
diff changeset
20
24142
21bd86e00bce PyNode_AddChild(): Backporting an aggressive over-allocation policy
Tim Peters <tim.peters@gmail.com>
parents: 14871
diff changeset
21 /* See comments at XXXROUNDUP below. Returns -1 on overflow. */
21bd86e00bce PyNode_AddChild(): Backporting an aggressive over-allocation policy
Tim Peters <tim.peters@gmail.com>
parents: 14871
diff changeset
22 static int
21bd86e00bce PyNode_AddChild(): Backporting an aggressive over-allocation policy
Tim Peters <tim.peters@gmail.com>
parents: 14871
diff changeset
23 fancy_roundup(int n)
21bd86e00bce PyNode_AddChild(): Backporting an aggressive over-allocation policy
Tim Peters <tim.peters@gmail.com>
parents: 14871
diff changeset
24 {
21bd86e00bce PyNode_AddChild(): Backporting an aggressive over-allocation policy
Tim Peters <tim.peters@gmail.com>
parents: 14871
diff changeset
25 /* Round up to the closest power of 2 >= n. */
21bd86e00bce PyNode_AddChild(): Backporting an aggressive over-allocation policy
Tim Peters <tim.peters@gmail.com>
parents: 14871
diff changeset
26 int result = 256;
21bd86e00bce PyNode_AddChild(): Backporting an aggressive over-allocation policy
Tim Peters <tim.peters@gmail.com>
parents: 14871
diff changeset
27 assert(n > 128);
21bd86e00bce PyNode_AddChild(): Backporting an aggressive over-allocation policy
Tim Peters <tim.peters@gmail.com>
parents: 14871
diff changeset
28 while (result < n) {
21bd86e00bce PyNode_AddChild(): Backporting an aggressive over-allocation policy
Tim Peters <tim.peters@gmail.com>
parents: 14871
diff changeset
29 result <<= 1;
21bd86e00bce PyNode_AddChild(): Backporting an aggressive over-allocation policy
Tim Peters <tim.peters@gmail.com>
parents: 14871
diff changeset
30 if (result <= 0)
21bd86e00bce PyNode_AddChild(): Backporting an aggressive over-allocation policy
Tim Peters <tim.peters@gmail.com>
parents: 14871
diff changeset
31 return -1;
21bd86e00bce PyNode_AddChild(): Backporting an aggressive over-allocation policy
Tim Peters <tim.peters@gmail.com>
parents: 14871
diff changeset
32 }
21bd86e00bce PyNode_AddChild(): Backporting an aggressive over-allocation policy
Tim Peters <tim.peters@gmail.com>
parents: 14871
diff changeset
33 return result;
21bd86e00bce PyNode_AddChild(): Backporting an aggressive over-allocation policy
Tim Peters <tim.peters@gmail.com>
parents: 14871
diff changeset
34 }
21bd86e00bce PyNode_AddChild(): Backporting an aggressive over-allocation policy
Tim Peters <tim.peters@gmail.com>
parents: 14871
diff changeset
35
21bd86e00bce PyNode_AddChild(): Backporting an aggressive over-allocation policy
Tim Peters <tim.peters@gmail.com>
parents: 14871
diff changeset
36 /* A gimmick to make massive numbers of reallocs quicker. The result is
21bd86e00bce PyNode_AddChild(): Backporting an aggressive over-allocation policy
Tim Peters <tim.peters@gmail.com>
parents: 14871
diff changeset
37 * a number >= the input. For n=0 we must return 0.
21bd86e00bce PyNode_AddChild(): Backporting an aggressive over-allocation policy
Tim Peters <tim.peters@gmail.com>
parents: 14871
diff changeset
38 * For n=1, we return 1, to avoid wasting memory in common 1-child nodes
21bd86e00bce PyNode_AddChild(): Backporting an aggressive over-allocation policy
Tim Peters <tim.peters@gmail.com>
parents: 14871
diff changeset
39 * (XXX are those actually common?).
21bd86e00bce PyNode_AddChild(): Backporting an aggressive over-allocation policy
Tim Peters <tim.peters@gmail.com>
parents: 14871
diff changeset
40 * Else for n <= 128, round up to the closest multiple of 4. Why 4?
21bd86e00bce PyNode_AddChild(): Backporting an aggressive over-allocation policy
Tim Peters <tim.peters@gmail.com>
parents: 14871
diff changeset
41 * Rounding up to a multiple of an exact power of 2 is very efficient.
21bd86e00bce PyNode_AddChild(): Backporting an aggressive over-allocation policy
Tim Peters <tim.peters@gmail.com>
parents: 14871
diff changeset
42 * Else call fancy_roundup() to grow proportionately to n. We've got an
21bd86e00bce PyNode_AddChild(): Backporting an aggressive over-allocation policy
Tim Peters <tim.peters@gmail.com>
parents: 14871
diff changeset
43 * extreme case then (like test_longexp.py), and on many platforms doing
21bd86e00bce PyNode_AddChild(): Backporting an aggressive over-allocation policy
Tim Peters <tim.peters@gmail.com>
parents: 14871
diff changeset
44 * anything less than proportional growth leads to exorbitant runtime
21bd86e00bce PyNode_AddChild(): Backporting an aggressive over-allocation policy
Tim Peters <tim.peters@gmail.com>
parents: 14871
diff changeset
45 * (e.g., MacPython), or extreme fragmentation of user address space (e.g.,
21bd86e00bce PyNode_AddChild(): Backporting an aggressive over-allocation policy
Tim Peters <tim.peters@gmail.com>
parents: 14871
diff changeset
46 * Win98).
21bd86e00bce PyNode_AddChild(): Backporting an aggressive over-allocation policy
Tim Peters <tim.peters@gmail.com>
parents: 14871
diff changeset
47 * This would be straightforward if a node stored its current capacity. The
21bd86e00bce PyNode_AddChild(): Backporting an aggressive over-allocation policy
Tim Peters <tim.peters@gmail.com>
parents: 14871
diff changeset
48 * code is tricky to avoid that.
21bd86e00bce PyNode_AddChild(): Backporting an aggressive over-allocation policy
Tim Peters <tim.peters@gmail.com>
parents: 14871
diff changeset
49 */
21bd86e00bce PyNode_AddChild(): Backporting an aggressive over-allocation policy
Tim Peters <tim.peters@gmail.com>
parents: 14871
diff changeset
50 #define XXXROUNDUP(n) ((n) == 1 ? 1 : \
21bd86e00bce PyNode_AddChild(): Backporting an aggressive over-allocation policy
Tim Peters <tim.peters@gmail.com>
parents: 14871
diff changeset
51 (n) <= 128 ? (((n) + 3) & ~3) : \
21bd86e00bce PyNode_AddChild(): Backporting an aggressive over-allocation policy
Tim Peters <tim.peters@gmail.com>
parents: 14871
diff changeset
52 fancy_roundup(n))
21bd86e00bce PyNode_AddChild(): Backporting an aggressive over-allocation policy
Tim Peters <tim.peters@gmail.com>
parents: 14871
diff changeset
53
6
fc6fcd7df4f7 Initial revision
Guido van Rossum <guido@python.org>
parents:
diff changeset
54
12970
63436719d12e Add new parser error code, E_OVERFLOW. This error is returned when
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 5379
diff changeset
55 int
13659
e78153e179fb Mass ANSIfication.
Thomas Wouters <thomas@python.org>
parents: 13447
diff changeset
56 PyNode_AddChild(register node *n1, int type, char *str, int lineno)
6
fc6fcd7df4f7 Initial revision
Guido van Rossum <guido@python.org>
parents:
diff changeset
57 {
24142
21bd86e00bce PyNode_AddChild(): Backporting an aggressive over-allocation policy
Tim Peters <tim.peters@gmail.com>
parents: 14871
diff changeset
58 const int nch = n1->n_nchildren;
21bd86e00bce PyNode_AddChild(): Backporting an aggressive over-allocation policy
Tim Peters <tim.peters@gmail.com>
parents: 14871
diff changeset
59 int current_capacity;
21bd86e00bce PyNode_AddChild(): Backporting an aggressive over-allocation policy
Tim Peters <tim.peters@gmail.com>
parents: 14871
diff changeset
60 int required_capacity;
21bd86e00bce PyNode_AddChild(): Backporting an aggressive over-allocation policy
Tim Peters <tim.peters@gmail.com>
parents: 14871
diff changeset
61 node *n;
21bd86e00bce PyNode_AddChild(): Backporting an aggressive over-allocation policy
Tim Peters <tim.peters@gmail.com>
parents: 14871
diff changeset
62
14147
21ec69a56228 Charles G. Waldman <cgw@fnal.gov>:
Fred Drake <fdrake@acm.org>
parents: 13659
diff changeset
63 if (nch == INT_MAX || nch < 0)
12970
63436719d12e Add new parser error code, E_OVERFLOW. This error is returned when
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 5379
diff changeset
64 return E_OVERFLOW;
24142
21bd86e00bce PyNode_AddChild(): Backporting an aggressive over-allocation policy
Tim Peters <tim.peters@gmail.com>
parents: 14871
diff changeset
65
21bd86e00bce PyNode_AddChild(): Backporting an aggressive over-allocation policy
Tim Peters <tim.peters@gmail.com>
parents: 14871
diff changeset
66 current_capacity = XXXROUNDUP(nch);
21bd86e00bce PyNode_AddChild(): Backporting an aggressive over-allocation policy
Tim Peters <tim.peters@gmail.com>
parents: 14871
diff changeset
67 required_capacity = XXXROUNDUP(nch + 1);
21bd86e00bce PyNode_AddChild(): Backporting an aggressive over-allocation policy
Tim Peters <tim.peters@gmail.com>
parents: 14871
diff changeset
68 if (current_capacity < 0 || required_capacity < 0)
21bd86e00bce PyNode_AddChild(): Backporting an aggressive over-allocation policy
Tim Peters <tim.peters@gmail.com>
parents: 14871
diff changeset
69 return E_OVERFLOW;
21bd86e00bce PyNode_AddChild(): Backporting an aggressive over-allocation policy
Tim Peters <tim.peters@gmail.com>
parents: 14871
diff changeset
70 if (current_capacity < required_capacity) {
6
fc6fcd7df4f7 Initial revision
Guido van Rossum <guido@python.org>
parents:
diff changeset
71 n = n1->n_child;
24142
21bd86e00bce PyNode_AddChild(): Backporting an aggressive over-allocation policy
Tim Peters <tim.peters@gmail.com>
parents: 14871
diff changeset
72 PyMem_RESIZE(n, node, required_capacity);
6
fc6fcd7df4f7 Initial revision
Guido van Rossum <guido@python.org>
parents:
diff changeset
73 if (n == NULL)
12970
63436719d12e Add new parser error code, E_OVERFLOW. This error is returned when
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 5379
diff changeset
74 return E_NOMEM;
6
fc6fcd7df4f7 Initial revision
Guido van Rossum <guido@python.org>
parents:
diff changeset
75 n1->n_child = n;
fc6fcd7df4f7 Initial revision
Guido van Rossum <guido@python.org>
parents:
diff changeset
76 }
24142
21bd86e00bce PyNode_AddChild(): Backporting an aggressive over-allocation policy
Tim Peters <tim.peters@gmail.com>
parents: 14871
diff changeset
77
6
fc6fcd7df4f7 Initial revision
Guido van Rossum <guido@python.org>
parents:
diff changeset
78 n = &n1->n_child[n1->n_nchildren++];
fc6fcd7df4f7 Initial revision
Guido van Rossum <guido@python.org>
parents:
diff changeset
79 n->n_type = type;
fc6fcd7df4f7 Initial revision
Guido van Rossum <guido@python.org>
parents:
diff changeset
80 n->n_str = str;
86
a6934380c6e7 "Compiling" version
Guido van Rossum <guido@python.org>
parents: 76
diff changeset
81 n->n_lineno = lineno;
6
fc6fcd7df4f7 Initial revision
Guido van Rossum <guido@python.org>
parents:
diff changeset
82 n->n_nchildren = 0;
fc6fcd7df4f7 Initial revision
Guido van Rossum <guido@python.org>
parents:
diff changeset
83 n->n_child = NULL;
12970
63436719d12e Add new parser error code, E_OVERFLOW. This error is returned when
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 5379
diff changeset
84 return 0;
6
fc6fcd7df4f7 Initial revision
Guido van Rossum <guido@python.org>
parents:
diff changeset
85 }
76
ede86799705b Add function to free an entire parse tree.
Guido van Rossum <guido@python.org>
parents: 6
diff changeset
86
86
a6934380c6e7 "Compiling" version
Guido van Rossum <guido@python.org>
parents: 76
diff changeset
87 /* Forward */
13447
a89f7f268de4 Nuke all remaining occurrences of Py_PROTO and Py_FPROTO.
Tim Peters <tim.peters@gmail.com>
parents: 13256
diff changeset
88 static void freechildren(node *);
86
a6934380c6e7 "Compiling" version
Guido van Rossum <guido@python.org>
parents: 76
diff changeset
89
a6934380c6e7 "Compiling" version
Guido van Rossum <guido@python.org>
parents: 76
diff changeset
90
a6934380c6e7 "Compiling" version
Guido van Rossum <guido@python.org>
parents: 76
diff changeset
91 void
13659
e78153e179fb Mass ANSIfication.
Thomas Wouters <thomas@python.org>
parents: 13447
diff changeset
92 PyNode_Free(node *n)
86
a6934380c6e7 "Compiling" version
Guido van Rossum <guido@python.org>
parents: 76
diff changeset
93 {
a6934380c6e7 "Compiling" version
Guido van Rossum <guido@python.org>
parents: 76
diff changeset
94 if (n != NULL) {
a6934380c6e7 "Compiling" version
Guido van Rossum <guido@python.org>
parents: 76
diff changeset
95 freechildren(n);
5379
1c9a84e0981d Another directory quickly renamed.
Guido van Rossum <guido@python.org>
parents: 4503
diff changeset
96 PyMem_DEL(n);
86
a6934380c6e7 "Compiling" version
Guido van Rossum <guido@python.org>
parents: 76
diff changeset
97 }
a6934380c6e7 "Compiling" version
Guido van Rossum <guido@python.org>
parents: 76
diff changeset
98 }
a6934380c6e7 "Compiling" version
Guido van Rossum <guido@python.org>
parents: 76
diff changeset
99
76
ede86799705b Add function to free an entire parse tree.
Guido van Rossum <guido@python.org>
parents: 6
diff changeset
100 static void
13659
e78153e179fb Mass ANSIfication.
Thomas Wouters <thomas@python.org>
parents: 13447
diff changeset
101 freechildren(node *n)
76
ede86799705b Add function to free an entire parse tree.
Guido van Rossum <guido@python.org>
parents: 6
diff changeset
102 {
ede86799705b Add function to free an entire parse tree.
Guido van Rossum <guido@python.org>
parents: 6
diff changeset
103 int i;
ede86799705b Add function to free an entire parse tree.
Guido van Rossum <guido@python.org>
parents: 6
diff changeset
104 for (i = NCH(n); --i >= 0; )
ede86799705b Add function to free an entire parse tree.
Guido van Rossum <guido@python.org>
parents: 6
diff changeset
105 freechildren(CHILD(n, i));
ede86799705b Add function to free an entire parse tree.
Guido van Rossum <guido@python.org>
parents: 6
diff changeset
106 if (n->n_child != NULL)
5379
1c9a84e0981d Another directory quickly renamed.
Guido van Rossum <guido@python.org>
parents: 4503
diff changeset
107 PyMem_DEL(n->n_child);
76
ede86799705b Add function to free an entire parse tree.
Guido van Rossum <guido@python.org>
parents: 6
diff changeset
108 if (STR(n) != NULL)
5379
1c9a84e0981d Another directory quickly renamed.
Guido van Rossum <guido@python.org>
parents: 4503
diff changeset
109 PyMem_DEL(STR(n));
76
ede86799705b Add function to free an entire parse tree.
Guido van Rossum <guido@python.org>
parents: 6
diff changeset
110 }