Mercurial > cpython-lru_cache-weakref

annotate Python/pyarena.c @ 57303:4aa4c19dfaed legacy-trunk

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
#1757126: fix typo with the cyrillic_asian alias
author Philip Jenvey <pjenvey@underboss.org>
date Tue, 17 Nov 2009 02:42:26 +0000
parents 12e973bdfeee
children fd25030223b0
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
36065
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
1 #include "Python.h"
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
2 #include "pyarena.h"
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
3
36813
54754f0d25c7 Document the purpose of the struct _block members.
Tim Peters <tim.peters@gmail.com>
parents: 36812
diff changeset
4 /* A simple arena block structure.
36728
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
5
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
6 Measurements with standard library modules suggest the average
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
7 allocation is about 20 bytes and that most compiles use a single
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
8 block.
37337
c9a6273775b5 Expand comments.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36813
diff changeset
9
c9a6273775b5 Expand comments.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36813
diff changeset
10 TODO(jhylton): Think about a realloc API, maybe just for the last
c9a6273775b5 Expand comments.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36813
diff changeset
11 allocation?
36728
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
12 */
36065
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
13
36716
71d873054d29 Real arena implementation
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36084
diff changeset
14 #define DEFAULT_BLOCK_SIZE 8192
37690
751f7c79b5fd Force 8-alignment of memory blocks, as needed on
Martin v. Löwis <martin@v.loewis.de>
parents: 37606
diff changeset
15 #define ALIGNMENT 8
751f7c79b5fd Force 8-alignment of memory blocks, as needed on
Martin v. Löwis <martin@v.loewis.de>
parents: 37606
diff changeset
16 #define ALIGNMENT_MASK (ALIGNMENT - 1)
751f7c79b5fd Force 8-alignment of memory blocks, as needed on
Martin v. Löwis <martin@v.loewis.de>
parents: 37606
diff changeset
17 #define ROUNDUP(x) (((x) + ALIGNMENT_MASK) & ~ALIGNMENT_MASK)
751f7c79b5fd Force 8-alignment of memory blocks, as needed on
Martin v. Löwis <martin@v.loewis.de>
parents: 37606
diff changeset
18
36716
71d873054d29 Real arena implementation
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36084
diff changeset
19 typedef struct _block {
36813
54754f0d25c7 Document the purpose of the struct _block members.
Tim Peters <tim.peters@gmail.com>
parents: 36812
diff changeset
20 /* Total number of bytes owned by this block available to pass out.
54754f0d25c7 Document the purpose of the struct _block members.
Tim Peters <tim.peters@gmail.com>
parents: 36812
diff changeset
21 * Read-only after initialization. The first such byte starts at
54754f0d25c7 Document the purpose of the struct _block members.
Tim Peters <tim.peters@gmail.com>
parents: 36812
diff changeset
22 * ab_mem.
54754f0d25c7 Document the purpose of the struct _block members.
Tim Peters <tim.peters@gmail.com>
parents: 36812
diff changeset
23 */
36720
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
24 size_t ab_size;
36813
54754f0d25c7 Document the purpose of the struct _block members.
Tim Peters <tim.peters@gmail.com>
parents: 36812
diff changeset
25
54754f0d25c7 Document the purpose of the struct _block members.
Tim Peters <tim.peters@gmail.com>
parents: 36812
diff changeset
26 /* Total number of bytes already passed out. The next byte available
54754f0d25c7 Document the purpose of the struct _block members.
Tim Peters <tim.peters@gmail.com>
parents: 36812
diff changeset
27 * to pass out starts at ab_mem + ab_offset.
54754f0d25c7 Document the purpose of the struct _block members.
Tim Peters <tim.peters@gmail.com>
parents: 36812
diff changeset
28 */
36720
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
29 size_t ab_offset;
36813
54754f0d25c7 Document the purpose of the struct _block members.
Tim Peters <tim.peters@gmail.com>
parents: 36812
diff changeset
30
54754f0d25c7 Document the purpose of the struct _block members.
Tim Peters <tim.peters@gmail.com>
parents: 36812
diff changeset
31 /* An arena maintains a singly-linked, NULL-terminated list of
54754f0d25c7 Document the purpose of the struct _block members.
Tim Peters <tim.peters@gmail.com>
parents: 36812
diff changeset
32 * all blocks owned by the arena. These are linked via the
54754f0d25c7 Document the purpose of the struct _block members.
Tim Peters <tim.peters@gmail.com>
parents: 36812
diff changeset
33 * ab_next member.
54754f0d25c7 Document the purpose of the struct _block members.
Tim Peters <tim.peters@gmail.com>
parents: 36812
diff changeset
34 */
36720
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
35 struct _block *ab_next;
36813
54754f0d25c7 Document the purpose of the struct _block members.
Tim Peters <tim.peters@gmail.com>
parents: 36812
diff changeset
36
54754f0d25c7 Document the purpose of the struct _block members.
Tim Peters <tim.peters@gmail.com>
parents: 36812
diff changeset
37 /* Pointer to the first allocatable byte owned by this block. Read-
54754f0d25c7 Document the purpose of the struct _block members.
Tim Peters <tim.peters@gmail.com>
parents: 36812
diff changeset
38 * only after initialization.
54754f0d25c7 Document the purpose of the struct _block members.
Tim Peters <tim.peters@gmail.com>
parents: 36812
diff changeset
39 */
36720
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
40 void *ab_mem;
36716
71d873054d29 Real arena implementation
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36084
diff changeset
41 } block;
36065
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
42
36724
5f3822c2e731 Use simple PyList to implement list of PyObject pointers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36720
diff changeset
43 /* The arena manages two kinds of memory, blocks of raw memory
5f3822c2e731 Use simple PyList to implement list of PyObject pointers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36720
diff changeset
44 and a list of PyObject* pointers. PyObjects are decrefed
5f3822c2e731 Use simple PyList to implement list of PyObject pointers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36720
diff changeset
45 when the arena is freed.
5f3822c2e731 Use simple PyList to implement list of PyObject pointers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36720
diff changeset
46 */
36808
b1a3c2238f97 Trimmed trailing whitespace.
Tim Peters <tim.peters@gmail.com>
parents: 36765
diff changeset
47
36065
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
48 struct _arena {
37337
c9a6273775b5 Expand comments.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36813
diff changeset
49 /* Pointer to the first block allocated for the arena, never NULL.
c9a6273775b5 Expand comments.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36813
diff changeset
50 It is used only to find the first block when the arena is
c9a6273775b5 Expand comments.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36813
diff changeset
51 being freed.
c9a6273775b5 Expand comments.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36813
diff changeset
52 */
36720
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
53 block *a_head;
37337
c9a6273775b5 Expand comments.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36813
diff changeset
54
c9a6273775b5 Expand comments.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36813
diff changeset
55 /* Pointer to the block currently used for allocation. It's
c9a6273775b5 Expand comments.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36813
diff changeset
56 ab_next field should be NULL. If it is not-null after a
c9a6273775b5 Expand comments.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36813
diff changeset
57 call to block_alloc(), it means a new block has been allocated
c9a6273775b5 Expand comments.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36813
diff changeset
58 and a_cur should be reset to point it.
c9a6273775b5 Expand comments.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36813
diff changeset
59 */
36720
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
60 block *a_cur;
37337
c9a6273775b5 Expand comments.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36813
diff changeset
61
c9a6273775b5 Expand comments.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36813
diff changeset
62 /* A Python list object containing references to all the PyObject
c9a6273775b5 Expand comments.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36813
diff changeset
63 pointers associated with this area. They will be DECREFed
c9a6273775b5 Expand comments.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36813
diff changeset
64 when the arena is freed.
c9a6273775b5 Expand comments.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36813
diff changeset
65 */
36724
5f3822c2e731 Use simple PyList to implement list of PyObject pointers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36720
diff changeset
66 PyObject *a_objects;
37337
c9a6273775b5 Expand comments.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36813
diff changeset
67
36728
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
68 #if defined(Py_DEBUG)
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
69 /* Debug output */
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
70 size_t total_allocs;
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
71 size_t total_size;
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
72 size_t total_blocks;
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
73 size_t total_block_size;
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
74 size_t total_big_blocks;
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
75 #endif
36065
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
76 };
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
77
36716
71d873054d29 Real arena implementation
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36084
diff changeset
78 static block *
71d873054d29 Real arena implementation
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36084
diff changeset
79 block_new(size_t size)
71d873054d29 Real arena implementation
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36084
diff changeset
80 {
36808
b1a3c2238f97 Trimmed trailing whitespace.
Tim Peters <tim.peters@gmail.com>
parents: 36765
diff changeset
81 /* Allocate header and block as one unit.
36720
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
82 ab_mem points just past header. */
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
83 block *b = (block *)malloc(sizeof(block) + size);
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
84 if (!b)
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
85 return NULL;
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
86 b->ab_size = size;
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
87 b->ab_mem = (void *)(b + 1);
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
88 b->ab_next = NULL;
37690
751f7c79b5fd Force 8-alignment of memory blocks, as needed on
Martin v. Löwis <martin@v.loewis.de>
parents: 37606
diff changeset
89 b->ab_offset = ROUNDUP((Py_uintptr_t)(b->ab_mem)) -
751f7c79b5fd Force 8-alignment of memory blocks, as needed on
Martin v. Löwis <martin@v.loewis.de>
parents: 37606
diff changeset
90 (Py_uintptr_t)(b->ab_mem);
36720
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
91 return b;
36065
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
92 }
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
93
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
94 static void
36716
71d873054d29 Real arena implementation
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36084
diff changeset
95 block_free(block *b) {
36720
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
96 while (b) {
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
97 block *next = b->ab_next;
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
98 free(b);
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
99 b = next;
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
100 }
36065
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
101 }
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
102
36716
71d873054d29 Real arena implementation
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36084
diff changeset
103 static void *
71d873054d29 Real arena implementation
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36084
diff changeset
104 block_alloc(block *b, size_t size)
71d873054d29 Real arena implementation
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36084
diff changeset
105 {
36720
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
106 void *p;
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
107 assert(b);
37690
751f7c79b5fd Force 8-alignment of memory blocks, as needed on
Martin v. Löwis <martin@v.loewis.de>
parents: 37606
diff changeset
108 size = ROUNDUP(size);
36720
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
109 if (b->ab_offset + size > b->ab_size) {
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
110 /* If we need to allocate more memory than will fit in
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
111 the default block, allocate a one-off block that is
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
112 exactly the right size. */
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
113 /* TODO(jhylton): Think about space waste at end of block */
37606
f448d0075691 more low-hanging fruit to make code compile under a C++ compiler. Not
Anthony Baxter <anthonybaxter@gmail.com>
parents: 37337
diff changeset
114 block *newbl = block_new(
36720
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
115 size < DEFAULT_BLOCK_SIZE ?
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
116 DEFAULT_BLOCK_SIZE : size);
37606
f448d0075691 more low-hanging fruit to make code compile under a C++ compiler. Not
Anthony Baxter <anthonybaxter@gmail.com>
parents: 37337
diff changeset
117 if (!newbl)
36720
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
118 return NULL;
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
119 assert(!b->ab_next);
37606
f448d0075691 more low-hanging fruit to make code compile under a C++ compiler. Not
Anthony Baxter <anthonybaxter@gmail.com>
parents: 37337
diff changeset
120 b->ab_next = newbl;
f448d0075691 more low-hanging fruit to make code compile under a C++ compiler. Not
Anthony Baxter <anthonybaxter@gmail.com>
parents: 37337
diff changeset
121 b = newbl;
36720
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
122 }
36716
71d873054d29 Real arena implementation
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36084
diff changeset
123
36720
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
124 assert(b->ab_offset + size <= b->ab_size);
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
125 p = (void *)(((char *)b->ab_mem) + b->ab_offset);
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
126 b->ab_offset += size;
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
127 return p;
36716
71d873054d29 Real arena implementation
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36084
diff changeset
128 }
36065
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
129
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
130 PyArena *
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
131 PyArena_New()
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
132 {
36720
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
133 PyArena* arena = (PyArena *)malloc(sizeof(PyArena));
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
134 if (!arena)
39135
12e973bdfeee Fix more memory allocation issues found with failmalloc.
Neal Norwitz <nnorwitz@gmail.com>
parents: 37746
diff changeset
135 return (PyArena*)PyErr_NoMemory();
36065
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
136
36720
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
137 arena->a_head = block_new(DEFAULT_BLOCK_SIZE);
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
138 arena->a_cur = arena->a_head;
36724
5f3822c2e731 Use simple PyList to implement list of PyObject pointers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36720
diff changeset
139 if (!arena->a_head) {
5f3822c2e731 Use simple PyList to implement list of PyObject pointers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36720
diff changeset
140 free((void *)arena);
39135
12e973bdfeee Fix more memory allocation issues found with failmalloc.
Neal Norwitz <nnorwitz@gmail.com>
parents: 37746
diff changeset
141 return (PyArena*)PyErr_NoMemory();
36724
5f3822c2e731 Use simple PyList to implement list of PyObject pointers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36720
diff changeset
142 }
36728
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
143 arena->a_objects = PyList_New(0);
36724
5f3822c2e731 Use simple PyList to implement list of PyObject pointers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36720
diff changeset
144 if (!arena->a_objects) {
5f3822c2e731 Use simple PyList to implement list of PyObject pointers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36720
diff changeset
145 block_free(arena->a_head);
5f3822c2e731 Use simple PyList to implement list of PyObject pointers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36720
diff changeset
146 free((void *)arena);
39135
12e973bdfeee Fix more memory allocation issues found with failmalloc.
Neal Norwitz <nnorwitz@gmail.com>
parents: 37746
diff changeset
147 return (PyArena*)PyErr_NoMemory();
36724
5f3822c2e731 Use simple PyList to implement list of PyObject pointers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36720
diff changeset
148 }
36728
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
149 #if defined(Py_DEBUG)
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
150 arena->total_allocs = 0;
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
151 arena->total_size = 0;
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
152 arena->total_blocks = 1;
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
153 arena->total_block_size = DEFAULT_BLOCK_SIZE;
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
154 arena->total_big_blocks = 0;
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
155 #endif
36720
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
156 return arena;
36065
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
157 }
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
158
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
159 void
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
160 PyArena_Free(PyArena *arena)
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
161 {
37337
c9a6273775b5 Expand comments.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36813
diff changeset
162 int r;
36720
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
163 assert(arena);
36728
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
164 #if defined(Py_DEBUG)
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
165 /*
36808
b1a3c2238f97 Trimmed trailing whitespace.
Tim Peters <tim.peters@gmail.com>
parents: 36765
diff changeset
166 fprintf(stderr,
36728
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
167 "alloc=%d size=%d blocks=%d block_size=%d big=%d objects=%d\n",
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
168 arena->total_allocs, arena->total_size, arena->total_blocks,
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
169 arena->total_block_size, arena->total_big_blocks,
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
170 PyList_Size(arena->a_objects));
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
171 */
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
172 #endif
36720
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
173 block_free(arena->a_head);
37746
636ca80a8cbe Fix sys.getobjects(0): we get a reference to the
Martin v. Löwis <martin@v.loewis.de>
parents: 37695
diff changeset
174 /* This property normally holds, except when the code being compiled
636ca80a8cbe Fix sys.getobjects(0): we get a reference to the
Martin v. Löwis <martin@v.loewis.de>
parents: 37695
diff changeset
175 is sys.getobjects(0), in which case there will be two references.
36724
5f3822c2e731 Use simple PyList to implement list of PyObject pointers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36720
diff changeset
176 assert(arena->a_objects->ob_refcnt == 1);
37746
636ca80a8cbe Fix sys.getobjects(0): we get a reference to the
Martin v. Löwis <martin@v.loewis.de>
parents: 37695
diff changeset
177 */
37337
c9a6273775b5 Expand comments.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36813
diff changeset
178
c9a6273775b5 Expand comments.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36813
diff changeset
179 /* Clear all the elements from the list. This is necessary
c9a6273775b5 Expand comments.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36813
diff changeset
180 to guarantee that they will be DECREFed. */
c9a6273775b5 Expand comments.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36813
diff changeset
181 r = PyList_SetSlice(arena->a_objects,
c9a6273775b5 Expand comments.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36813
diff changeset
182 0, PyList_GET_SIZE(arena->a_objects), NULL);
c9a6273775b5 Expand comments.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36813
diff changeset
183 assert(r == 0);
c9a6273775b5 Expand comments.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36813
diff changeset
184 assert(PyList_GET_SIZE(arena->a_objects) == 0);
36724
5f3822c2e731 Use simple PyList to implement list of PyObject pointers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36720
diff changeset
185 Py_DECREF(arena->a_objects);
36720
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
186 free(arena);
36065
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
187 }
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
188
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
189 void *
36808
b1a3c2238f97 Trimmed trailing whitespace.
Tim Peters <tim.peters@gmail.com>
parents: 36765
diff changeset
190 PyArena_Malloc(PyArena *arena, size_t size)
36065
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
191 {
36720
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
192 void *p = block_alloc(arena->a_cur, size);
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
193 if (!p)
39135
12e973bdfeee Fix more memory allocation issues found with failmalloc.
Neal Norwitz <nnorwitz@gmail.com>
parents: 37746
diff changeset
194 return PyErr_NoMemory();
36728
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
195 #if defined(Py_DEBUG)
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
196 arena->total_allocs++;
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
197 arena->total_size += size;
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
198 #endif
36720
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
199 /* Reset cur if we allocated a new block. */
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
200 if (arena->a_cur->ab_next) {
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
201 arena->a_cur = arena->a_cur->ab_next;
36728
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
202 #if defined(Py_DEBUG)
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
203 arena->total_blocks++;
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
204 arena->total_block_size += arena->a_cur->ab_size;
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
205 if (arena->a_cur->ab_size > DEFAULT_BLOCK_SIZE)
36812
1d055ebd8322 Added words about what PyArena_Malloc() does.
Tim Peters <tim.peters@gmail.com>
parents: 36808
diff changeset
206 ++arena->total_big_blocks;
36728
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
207 #endif
36720
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
208 }
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
209 return p;
36065
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
210 }
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
211
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
212 int
36808
b1a3c2238f97 Trimmed trailing whitespace.
Tim Peters <tim.peters@gmail.com>
parents: 36765
diff changeset
213 PyArena_AddPyObject(PyArena *arena, PyObject *obj)
36065
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
214 {
36765
0674796b696c Add missing DECREF.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36728
diff changeset
215 int r = PyList_Append(arena->a_objects, obj);
0674796b696c Add missing DECREF.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36728
diff changeset
216 if (r >= 0) {
0674796b696c Add missing DECREF.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36728
diff changeset
217 Py_DECREF(obj);
0674796b696c Add missing DECREF.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36728
diff changeset
218 }
0674796b696c Add missing DECREF.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36728
diff changeset
219 return r;
36065
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
220 }