Mercurial > cpython-lru_cache-weakref

annotate Python/pyarena.c @ 49955:878d40468c4e legacy-trunk

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Issue #4384: Added logging integration with warnings module using captureWarnings(). This change includes a NullHandler which does nothing; it will be of use to library developers who want to avoid the "No handlers could be found for logger XXX" message which can appear if the library user doesn't configure logging.
author Vinay Sajip <vinay_sajip@yahoo.co.uk>
date Wed, 03 Dec 2008 23:22:58 +0000
parents 12e973bdfeee
children fd25030223b0
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
36065
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
1 #include "Python.h"
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
2 #include "pyarena.h"
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
3
36813
54754f0d25c7 Document the purpose of the struct _block members.
Tim Peters <tim.peters@gmail.com>
parents: 36812
diff changeset
4 /* A simple arena block structure.
36728
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
5
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
6 Measurements with standard library modules suggest the average
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
7 allocation is about 20 bytes and that most compiles use a single
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
8 block.
37337
c9a6273775b5 Expand comments.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36813
diff changeset
9
c9a6273775b5 Expand comments.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36813
diff changeset
10 TODO(jhylton): Think about a realloc API, maybe just for the last
c9a6273775b5 Expand comments.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36813
diff changeset
11 allocation?
36728
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
12 */
36065
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
13
36716
71d873054d29 Real arena implementation
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36084
diff changeset
14 #define DEFAULT_BLOCK_SIZE 8192
37690
751f7c79b5fd Force 8-alignment of memory blocks, as needed on
Martin v. Löwis <martin@v.loewis.de>
parents: 37606
diff changeset
15 #define ALIGNMENT 8
751f7c79b5fd Force 8-alignment of memory blocks, as needed on
Martin v. Löwis <martin@v.loewis.de>
parents: 37606
diff changeset
16 #define ALIGNMENT_MASK (ALIGNMENT - 1)
751f7c79b5fd Force 8-alignment of memory blocks, as needed on
Martin v. Löwis <martin@v.loewis.de>
parents: 37606
diff changeset
17 #define ROUNDUP(x) (((x) + ALIGNMENT_MASK) & ~ALIGNMENT_MASK)
751f7c79b5fd Force 8-alignment of memory blocks, as needed on
Martin v. Löwis <martin@v.loewis.de>
parents: 37606
diff changeset
18
36716
71d873054d29 Real arena implementation
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36084
diff changeset
19 typedef struct _block {
36813
54754f0d25c7 Document the purpose of the struct _block members.
Tim Peters <tim.peters@gmail.com>
parents: 36812
diff changeset
20 /* Total number of bytes owned by this block available to pass out.
54754f0d25c7 Document the purpose of the struct _block members.
Tim Peters <tim.peters@gmail.com>
parents: 36812
diff changeset
21 * Read-only after initialization. The first such byte starts at
54754f0d25c7 Document the purpose of the struct _block members.
Tim Peters <tim.peters@gmail.com>
parents: 36812
diff changeset
22 * ab_mem.
54754f0d25c7 Document the purpose of the struct _block members.
Tim Peters <tim.peters@gmail.com>
parents: 36812
diff changeset
23 */
36720
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
24 size_t ab_size;
36813
54754f0d25c7 Document the purpose of the struct _block members.
Tim Peters <tim.peters@gmail.com>
parents: 36812
diff changeset
25
54754f0d25c7 Document the purpose of the struct _block members.
Tim Peters <tim.peters@gmail.com>
parents: 36812
diff changeset
26 /* Total number of bytes already passed out. The next byte available
54754f0d25c7 Document the purpose of the struct _block members.
Tim Peters <tim.peters@gmail.com>
parents: 36812
diff changeset
27 * to pass out starts at ab_mem + ab_offset.
54754f0d25c7 Document the purpose of the struct _block members.
Tim Peters <tim.peters@gmail.com>
parents: 36812
diff changeset
28 */
36720
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
29 size_t ab_offset;
36813
54754f0d25c7 Document the purpose of the struct _block members.
Tim Peters <tim.peters@gmail.com>
parents: 36812
diff changeset
30
54754f0d25c7 Document the purpose of the struct _block members.
Tim Peters <tim.peters@gmail.com>
parents: 36812
diff changeset
31 /* An arena maintains a singly-linked, NULL-terminated list of
54754f0d25c7 Document the purpose of the struct _block members.
Tim Peters <tim.peters@gmail.com>
parents: 36812
diff changeset
32 * all blocks owned by the arena. These are linked via the
54754f0d25c7 Document the purpose of the struct _block members.
Tim Peters <tim.peters@gmail.com>
parents: 36812
diff changeset
33 * ab_next member.
54754f0d25c7 Document the purpose of the struct _block members.
Tim Peters <tim.peters@gmail.com>
parents: 36812
diff changeset
34 */
36720
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
35 struct _block *ab_next;
36813
54754f0d25c7 Document the purpose of the struct _block members.
Tim Peters <tim.peters@gmail.com>
parents: 36812
diff changeset
36
54754f0d25c7 Document the purpose of the struct _block members.
Tim Peters <tim.peters@gmail.com>
parents: 36812
diff changeset
37 /* Pointer to the first allocatable byte owned by this block. Read-
54754f0d25c7 Document the purpose of the struct _block members.
Tim Peters <tim.peters@gmail.com>
parents: 36812
diff changeset
38 * only after initialization.
54754f0d25c7 Document the purpose of the struct _block members.
Tim Peters <tim.peters@gmail.com>
parents: 36812
diff changeset
39 */
36720
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
40 void *ab_mem;
36716
71d873054d29 Real arena implementation
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36084
diff changeset
41 } block;
36065
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
42
36724
5f3822c2e731 Use simple PyList to implement list of PyObject pointers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36720
diff changeset
43 /* The arena manages two kinds of memory, blocks of raw memory
5f3822c2e731 Use simple PyList to implement list of PyObject pointers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36720
diff changeset
44 and a list of PyObject* pointers. PyObjects are decrefed
5f3822c2e731 Use simple PyList to implement list of PyObject pointers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36720
diff changeset
45 when the arena is freed.
5f3822c2e731 Use simple PyList to implement list of PyObject pointers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36720
diff changeset
46 */
36808
b1a3c2238f97 Trimmed trailing whitespace.
Tim Peters <tim.peters@gmail.com>
parents: 36765
diff changeset
47
36065
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
48 struct _arena {
37337
c9a6273775b5 Expand comments.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36813
diff changeset
49 /* Pointer to the first block allocated for the arena, never NULL.
c9a6273775b5 Expand comments.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36813
diff changeset
50 It is used only to find the first block when the arena is
c9a6273775b5 Expand comments.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36813
diff changeset
51 being freed.
c9a6273775b5 Expand comments.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36813
diff changeset
52 */
36720
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
53 block *a_head;
37337
c9a6273775b5 Expand comments.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36813
diff changeset
54
c9a6273775b5 Expand comments.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36813
diff changeset
55 /* Pointer to the block currently used for allocation. It's
c9a6273775b5 Expand comments.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36813
diff changeset
56 ab_next field should be NULL. If it is not-null after a
c9a6273775b5 Expand comments.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36813
diff changeset
57 call to block_alloc(), it means a new block has been allocated
c9a6273775b5 Expand comments.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36813
diff changeset
58 and a_cur should be reset to point it.
c9a6273775b5 Expand comments.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36813
diff changeset
59 */
36720
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
60 block *a_cur;
37337
c9a6273775b5 Expand comments.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36813
diff changeset
61
c9a6273775b5 Expand comments.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36813
diff changeset
62 /* A Python list object containing references to all the PyObject
c9a6273775b5 Expand comments.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36813
diff changeset
63 pointers associated with this area. They will be DECREFed
c9a6273775b5 Expand comments.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36813
diff changeset
64 when the arena is freed.
c9a6273775b5 Expand comments.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36813
diff changeset
65 */
36724
5f3822c2e731 Use simple PyList to implement list of PyObject pointers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36720
diff changeset
66 PyObject *a_objects;
37337
c9a6273775b5 Expand comments.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36813
diff changeset
67
36728
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
68 #if defined(Py_DEBUG)
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
69 /* Debug output */
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
70 size_t total_allocs;
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
71 size_t total_size;
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
72 size_t total_blocks;
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
73 size_t total_block_size;
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
74 size_t total_big_blocks;
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
75 #endif
36065
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
76 };
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
77
36716
71d873054d29 Real arena implementation
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36084
diff changeset
78 static block *
71d873054d29 Real arena implementation
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36084
diff changeset
79 block_new(size_t size)
71d873054d29 Real arena implementation
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36084
diff changeset
80 {
36808
b1a3c2238f97 Trimmed trailing whitespace.
Tim Peters <tim.peters@gmail.com>
parents: 36765
diff changeset
81 /* Allocate header and block as one unit.
36720
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
82 ab_mem points just past header. */
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
83 block *b = (block *)malloc(sizeof(block) + size);
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
84 if (!b)
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
85 return NULL;
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
86 b->ab_size = size;
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
87 b->ab_mem = (void *)(b + 1);
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
88 b->ab_next = NULL;
37690
751f7c79b5fd Force 8-alignment of memory blocks, as needed on
Martin v. Löwis <martin@v.loewis.de>
parents: 37606
diff changeset
89 b->ab_offset = ROUNDUP((Py_uintptr_t)(b->ab_mem)) -
751f7c79b5fd Force 8-alignment of memory blocks, as needed on
Martin v. Löwis <martin@v.loewis.de>
parents: 37606
diff changeset
90 (Py_uintptr_t)(b->ab_mem);
36720
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
91 return b;
36065
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
92 }
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
93
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
94 static void
36716
71d873054d29 Real arena implementation
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36084
diff changeset
95 block_free(block *b) {
36720
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
96 while (b) {
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
97 block *next = b->ab_next;
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
98 free(b);
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
99 b = next;
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
100 }
36065
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
101 }
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
102
36716
71d873054d29 Real arena implementation
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36084
diff changeset
103 static void *
71d873054d29 Real arena implementation
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36084
diff changeset
104 block_alloc(block *b, size_t size)
71d873054d29 Real arena implementation
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36084
diff changeset
105 {
36720
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
106 void *p;
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
107 assert(b);
37690
751f7c79b5fd Force 8-alignment of memory blocks, as needed on
Martin v. Löwis <martin@v.loewis.de>
parents: 37606
diff changeset
108 size = ROUNDUP(size);
36720
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
109 if (b->ab_offset + size > b->ab_size) {
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
110 /* If we need to allocate more memory than will fit in
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
111 the default block, allocate a one-off block that is
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
112 exactly the right size. */
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
113 /* TODO(jhylton): Think about space waste at end of block */
37606
f448d0075691 more low-hanging fruit to make code compile under a C++ compiler. Not
Anthony Baxter <anthonybaxter@gmail.com>
parents: 37337
diff changeset
114 block *newbl = block_new(
36720
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
115 size < DEFAULT_BLOCK_SIZE ?
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
116 DEFAULT_BLOCK_SIZE : size);
37606
f448d0075691 more low-hanging fruit to make code compile under a C++ compiler. Not
Anthony Baxter <anthonybaxter@gmail.com>
parents: 37337
diff changeset
117 if (!newbl)
36720
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
118 return NULL;
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
119 assert(!b->ab_next);
37606
f448d0075691 more low-hanging fruit to make code compile under a C++ compiler. Not
Anthony Baxter <anthonybaxter@gmail.com>
parents: 37337
diff changeset
120 b->ab_next = newbl;
f448d0075691 more low-hanging fruit to make code compile under a C++ compiler. Not
Anthony Baxter <anthonybaxter@gmail.com>
parents: 37337
diff changeset
121 b = newbl;
36720
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
122 }
36716
71d873054d29 Real arena implementation
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36084
diff changeset
123
36720
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
124 assert(b->ab_offset + size <= b->ab_size);
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
125 p = (void *)(((char *)b->ab_mem) + b->ab_offset);
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
126 b->ab_offset += size;
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
127 return p;
36716
71d873054d29 Real arena implementation
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36084
diff changeset
128 }
36065
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
129
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
130 PyArena *
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
131 PyArena_New()
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
132 {
36720
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
133 PyArena* arena = (PyArena *)malloc(sizeof(PyArena));
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
134 if (!arena)
39135
12e973bdfeee Fix more memory allocation issues found with failmalloc.
Neal Norwitz <nnorwitz@gmail.com>
parents: 37746
diff changeset
135 return (PyArena*)PyErr_NoMemory();
36065
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
136
36720
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
137 arena->a_head = block_new(DEFAULT_BLOCK_SIZE);
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
138 arena->a_cur = arena->a_head;
36724
5f3822c2e731 Use simple PyList to implement list of PyObject pointers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36720
diff changeset
139 if (!arena->a_head) {
5f3822c2e731 Use simple PyList to implement list of PyObject pointers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36720
diff changeset
140 free((void *)arena);
39135
12e973bdfeee Fix more memory allocation issues found with failmalloc.
Neal Norwitz <nnorwitz@gmail.com>
parents: 37746
diff changeset
141 return (PyArena*)PyErr_NoMemory();
36724
5f3822c2e731 Use simple PyList to implement list of PyObject pointers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36720
diff changeset
142 }
36728
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
143 arena->a_objects = PyList_New(0);
36724
5f3822c2e731 Use simple PyList to implement list of PyObject pointers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36720
diff changeset
144 if (!arena->a_objects) {
5f3822c2e731 Use simple PyList to implement list of PyObject pointers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36720
diff changeset
145 block_free(arena->a_head);
5f3822c2e731 Use simple PyList to implement list of PyObject pointers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36720
diff changeset
146 free((void *)arena);
39135
12e973bdfeee Fix more memory allocation issues found with failmalloc.
Neal Norwitz <nnorwitz@gmail.com>
parents: 37746
diff changeset
147 return (PyArena*)PyErr_NoMemory();
36724
5f3822c2e731 Use simple PyList to implement list of PyObject pointers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36720
diff changeset
148 }
36728
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
149 #if defined(Py_DEBUG)
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
150 arena->total_allocs = 0;
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
151 arena->total_size = 0;
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
152 arena->total_blocks = 1;
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
153 arena->total_block_size = DEFAULT_BLOCK_SIZE;
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
154 arena->total_big_blocks = 0;
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
155 #endif
36720
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
156 return arena;
36065
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
157 }
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
158
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
159 void
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
160 PyArena_Free(PyArena *arena)
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
161 {
37337
c9a6273775b5 Expand comments.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36813
diff changeset
162 int r;
36720
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
163 assert(arena);
36728
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
164 #if defined(Py_DEBUG)
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
165 /*
36808
b1a3c2238f97 Trimmed trailing whitespace.
Tim Peters <tim.peters@gmail.com>
parents: 36765
diff changeset
166 fprintf(stderr,
36728
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
167 "alloc=%d size=%d blocks=%d block_size=%d big=%d objects=%d\n",
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
168 arena->total_allocs, arena->total_size, arena->total_blocks,
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
169 arena->total_block_size, arena->total_big_blocks,
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
170 PyList_Size(arena->a_objects));
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
171 */
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
172 #endif
36720
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
173 block_free(arena->a_head);
37746
636ca80a8cbe Fix sys.getobjects(0): we get a reference to the
Martin v. Löwis <martin@v.loewis.de>
parents: 37695
diff changeset
174 /* This property normally holds, except when the code being compiled
636ca80a8cbe Fix sys.getobjects(0): we get a reference to the
Martin v. Löwis <martin@v.loewis.de>
parents: 37695
diff changeset
175 is sys.getobjects(0), in which case there will be two references.
36724
5f3822c2e731 Use simple PyList to implement list of PyObject pointers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36720
diff changeset
176 assert(arena->a_objects->ob_refcnt == 1);
37746
636ca80a8cbe Fix sys.getobjects(0): we get a reference to the
Martin v. Löwis <martin@v.loewis.de>
parents: 37695
diff changeset
177 */
37337
c9a6273775b5 Expand comments.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36813
diff changeset
178
c9a6273775b5 Expand comments.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36813
diff changeset
179 /* Clear all the elements from the list. This is necessary
c9a6273775b5 Expand comments.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36813
diff changeset
180 to guarantee that they will be DECREFed. */
c9a6273775b5 Expand comments.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36813
diff changeset
181 r = PyList_SetSlice(arena->a_objects,
c9a6273775b5 Expand comments.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36813
diff changeset
182 0, PyList_GET_SIZE(arena->a_objects), NULL);
c9a6273775b5 Expand comments.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36813
diff changeset
183 assert(r == 0);
c9a6273775b5 Expand comments.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36813
diff changeset
184 assert(PyList_GET_SIZE(arena->a_objects) == 0);
36724
5f3822c2e731 Use simple PyList to implement list of PyObject pointers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36720
diff changeset
185 Py_DECREF(arena->a_objects);
36720
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
186 free(arena);
36065
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
187 }
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
188
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
189 void *
36808
b1a3c2238f97 Trimmed trailing whitespace.
Tim Peters <tim.peters@gmail.com>
parents: 36765
diff changeset
190 PyArena_Malloc(PyArena *arena, size_t size)
36065
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
191 {
36720
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
192 void *p = block_alloc(arena->a_cur, size);
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
193 if (!p)
39135
12e973bdfeee Fix more memory allocation issues found with failmalloc.
Neal Norwitz <nnorwitz@gmail.com>
parents: 37746
diff changeset
194 return PyErr_NoMemory();
36728
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
195 #if defined(Py_DEBUG)
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
196 arena->total_allocs++;
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
197 arena->total_size += size;
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
198 #endif
36720
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
199 /* Reset cur if we allocated a new block. */
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
200 if (arena->a_cur->ab_next) {
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
201 arena->a_cur = arena->a_cur->ab_next;
36728
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
202 #if defined(Py_DEBUG)
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
203 arena->total_blocks++;
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
204 arena->total_block_size += arena->a_cur->ab_size;
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
205 if (arena->a_cur->ab_size > DEFAULT_BLOCK_SIZE)
36812
1d055ebd8322 Added words about what PyArena_Malloc() does.
Tim Peters <tim.peters@gmail.com>
parents: 36808
diff changeset
206 ++arena->total_big_blocks;
36728
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
207 #endif
36720
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
208 }
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36716
diff changeset
209 return p;
36065
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
210 }
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
211
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
212 int
36808
b1a3c2238f97 Trimmed trailing whitespace.
Tim Peters <tim.peters@gmail.com>
parents: 36765
diff changeset
213 PyArena_AddPyObject(PyArena *arena, PyObject *obj)
36065
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
214 {
36765
0674796b696c Add missing DECREF.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36728
diff changeset
215 int r = PyList_Append(arena->a_objects, obj);
0674796b696c Add missing DECREF.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36728
diff changeset
216 if (r >= 0) {
0674796b696c Add missing DECREF.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36728
diff changeset
217 Py_DECREF(obj);
0674796b696c Add missing DECREF.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36728
diff changeset
218 }
0674796b696c Add missing DECREF.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36728
diff changeset
219 return r;
36065
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
220 }