annotate Python/pyarena.c @ 73323:dad02a080bbc

Improve byte-compilation in packaging to be independent of -O or -B. The code I fixed to comply with PEP 3147 still had one bug: When run under python -O, some paths for pyc files would be pyo, because I called imp.cache_from_source without explicit debug_override argument in some places, and under -O that would return .pyo (this is well explained in the imp docs). Now all code (util.byte_compile, build_py, install_lib) can create .pyo files according to options given by users, without interference from the calling Python’s own optimize mode. On a related topic, I also removed the code that prevented byte compilation under python -B. The rationale is that packaging gives control over the creation of pyc files to the user with its own explicit option, and the behavior should not be changed if the calling Python happens to run with -B for whatever reason. I will argue that this is a bug fix and ask to be allowed to backport this change to distutils. Finally, I moved one nugget of information about the --compile and --optimize options from the source into the doc. It clears up a misunderstanding that I (and maybe other people) had.
author Éric Araujo <merwok@netwok.org>
date Thu, 03 Nov 2011 05:08:28 +0100
parents 724920a015f7
children 99112b851b25
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
36065
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
1 #include "Python.h"
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
2
36813
54754f0d25c7 Document the purpose of the struct _block members.
Tim Peters <tim.peters@gmail.com>
parents: 36812
diff changeset
3 /* A simple arena block structure.
36728
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
4
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
5 Measurements with standard library modules suggest the average
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
6 allocation is about 20 bytes and that most compiles use a single
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
7 block.
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 36813
diff changeset
8
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 36813
diff changeset
9 TODO(jhylton): Think about a realloc API, maybe just for the last
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 36813
diff changeset
10 allocation?
36728
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
11 */
36065
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
12
36716
71d873054d29 Real arena implementation
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36084
diff changeset
13 #define DEFAULT_BLOCK_SIZE 8192
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
14 #define ALIGNMENT 8
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
15 #define ALIGNMENT_MASK (ALIGNMENT - 1)
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
16 #define ROUNDUP(x) (((x) + ALIGNMENT_MASK) & ~ALIGNMENT_MASK)
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 36813
diff changeset
17
36716
71d873054d29 Real arena implementation
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36084
diff changeset
18 typedef struct _block {
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
19 /* Total number of bytes owned by this block available to pass out.
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
20 * Read-only after initialization. The first such byte starts at
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
21 * ab_mem.
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
22 */
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
23 size_t ab_size;
36813
54754f0d25c7 Document the purpose of the struct _block members.
Tim Peters <tim.peters@gmail.com>
parents: 36812
diff changeset
24
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
25 /* Total number of bytes already passed out. The next byte available
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
26 * to pass out starts at ab_mem + ab_offset.
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
27 */
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
28 size_t ab_offset;
36813
54754f0d25c7 Document the purpose of the struct _block members.
Tim Peters <tim.peters@gmail.com>
parents: 36812
diff changeset
29
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
30 /* An arena maintains a singly-linked, NULL-terminated list of
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
31 * all blocks owned by the arena. These are linked via the
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
32 * ab_next member.
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
33 */
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
34 struct _block *ab_next;
36813
54754f0d25c7 Document the purpose of the struct _block members.
Tim Peters <tim.peters@gmail.com>
parents: 36812
diff changeset
35
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
36 /* Pointer to the first allocatable byte owned by this block. Read-
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
37 * only after initialization.
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
38 */
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
39 void *ab_mem;
36716
71d873054d29 Real arena implementation
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36084
diff changeset
40 } block;
36065
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
41
36724
5f3822c2e731 Use simple PyList to implement list of PyObject pointers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36720
diff changeset
42 /* The arena manages two kinds of memory, blocks of raw memory
5f3822c2e731 Use simple PyList to implement list of PyObject pointers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36720
diff changeset
43 and a list of PyObject* pointers. PyObjects are decrefed
5f3822c2e731 Use simple PyList to implement list of PyObject pointers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36720
diff changeset
44 when the arena is freed.
5f3822c2e731 Use simple PyList to implement list of PyObject pointers
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36720
diff changeset
45 */
36808
b1a3c2238f97 Trimmed trailing whitespace.
Tim Peters <tim.peters@gmail.com>
parents: 36765
diff changeset
46
36065
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
47 struct _arena {
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
48 /* Pointer to the first block allocated for the arena, never NULL.
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
49 It is used only to find the first block when the arena is
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
50 being freed.
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
51 */
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
52 block *a_head;
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 36813
diff changeset
53
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
54 /* Pointer to the block currently used for allocation. It's
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
55 ab_next field should be NULL. If it is not-null after a
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
56 call to block_alloc(), it means a new block has been allocated
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
57 and a_cur should be reset to point it.
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
58 */
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
59 block *a_cur;
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 36813
diff changeset
60
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
61 /* A Python list object containing references to all the PyObject
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
62 pointers associated with this area. They will be DECREFed
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
63 when the arena is freed.
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
64 */
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
65 PyObject *a_objects;
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 36813
diff changeset
66
36728
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
67 #if defined(Py_DEBUG)
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
68 /* Debug output */
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
69 size_t total_allocs;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
70 size_t total_size;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
71 size_t total_blocks;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
72 size_t total_block_size;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
73 size_t total_big_blocks;
36728
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
74 #endif
36065
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
75 };
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
76
36716
71d873054d29 Real arena implementation
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36084
diff changeset
77 static block *
71d873054d29 Real arena implementation
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36084
diff changeset
78 block_new(size_t size)
71d873054d29 Real arena implementation
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36084
diff changeset
79 {
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
80 /* Allocate header and block as one unit.
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
81 ab_mem points just past header. */
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
82 block *b = (block *)malloc(sizeof(block) + size);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
83 if (!b)
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
84 return NULL;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
85 b->ab_size = size;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
86 b->ab_mem = (void *)(b + 1);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
87 b->ab_next = NULL;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
88 b->ab_offset = ROUNDUP((Py_uintptr_t)(b->ab_mem)) -
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
89 (Py_uintptr_t)(b->ab_mem);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
90 return b;
36065
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
91 }
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
92
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
93 static void
36716
71d873054d29 Real arena implementation
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36084
diff changeset
94 block_free(block *b) {
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
95 while (b) {
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
96 block *next = b->ab_next;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
97 free(b);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
98 b = next;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
99 }
36065
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
100 }
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
101
36716
71d873054d29 Real arena implementation
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36084
diff changeset
102 static void *
71d873054d29 Real arena implementation
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36084
diff changeset
103 block_alloc(block *b, size_t size)
71d873054d29 Real arena implementation
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36084
diff changeset
104 {
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
105 void *p;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
106 assert(b);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
107 size = ROUNDUP(size);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
108 if (b->ab_offset + size > b->ab_size) {
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
109 /* If we need to allocate more memory than will fit in
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
110 the default block, allocate a one-off block that is
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
111 exactly the right size. */
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
112 /* TODO(jhylton): Think about space waste at end of block */
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
113 block *newbl = block_new(
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
114 size < DEFAULT_BLOCK_SIZE ?
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
115 DEFAULT_BLOCK_SIZE : size);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
116 if (!newbl)
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
117 return NULL;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
118 assert(!b->ab_next);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
119 b->ab_next = newbl;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
120 b = newbl;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
121 }
36716
71d873054d29 Real arena implementation
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36084
diff changeset
122
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
123 assert(b->ab_offset + size <= b->ab_size);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
124 p = (void *)(((char *)b->ab_mem) + b->ab_offset);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
125 b->ab_offset += size;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
126 return p;
36716
71d873054d29 Real arena implementation
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36084
diff changeset
127 }
36065
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
128
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
129 PyArena *
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
130 PyArena_New()
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
131 {
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
132 PyArena* arena = (PyArena *)malloc(sizeof(PyArena));
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
133 if (!arena)
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
134 return (PyArena*)PyErr_NoMemory();
36065
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
135
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
136 arena->a_head = block_new(DEFAULT_BLOCK_SIZE);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
137 arena->a_cur = arena->a_head;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
138 if (!arena->a_head) {
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
139 free((void *)arena);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
140 return (PyArena*)PyErr_NoMemory();
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
141 }
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
142 arena->a_objects = PyList_New(0);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
143 if (!arena->a_objects) {
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
144 block_free(arena->a_head);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
145 free((void *)arena);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
146 return (PyArena*)PyErr_NoMemory();
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
147 }
36728
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
148 #if defined(Py_DEBUG)
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
149 arena->total_allocs = 0;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
150 arena->total_size = 0;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
151 arena->total_blocks = 1;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
152 arena->total_block_size = DEFAULT_BLOCK_SIZE;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
153 arena->total_big_blocks = 0;
36728
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
154 #endif
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
155 return arena;
36065
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
156 }
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
157
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
158 void
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
159 PyArena_Free(PyArena *arena)
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
160 {
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
161 int r;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
162 assert(arena);
36728
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
163 #if defined(Py_DEBUG)
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
164 /*
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
165 fprintf(stderr,
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
166 "alloc=%d size=%d blocks=%d block_size=%d big=%d objects=%d\n",
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
167 arena->total_allocs, arena->total_size, arena->total_blocks,
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
168 arena->total_block_size, arena->total_big_blocks,
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
169 PyList_Size(arena->a_objects));
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
170 */
36728
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
171 #endif
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
172 block_free(arena->a_head);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
173 /* This property normally holds, except when the code being compiled
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
174 is sys.getobjects(0), in which case there will be two references.
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
175 assert(arena->a_objects->ob_refcnt == 1);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
176 */
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 36813
diff changeset
177
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
178 /* Clear all the elements from the list. This is necessary
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
179 to guarantee that they will be DECREFed. */
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
180 r = PyList_SetSlice(arena->a_objects,
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
181 0, PyList_GET_SIZE(arena->a_objects), NULL);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
182 assert(r == 0);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
183 assert(PyList_GET_SIZE(arena->a_objects) == 0);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
184 Py_DECREF(arena->a_objects);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
185 free(arena);
36065
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
186 }
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
187
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
188 void *
36808
b1a3c2238f97 Trimmed trailing whitespace.
Tim Peters <tim.peters@gmail.com>
parents: 36765
diff changeset
189 PyArena_Malloc(PyArena *arena, size_t size)
36065
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
190 {
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
191 void *p = block_alloc(arena->a_cur, size);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
192 if (!p)
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
193 return PyErr_NoMemory();
36728
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
194 #if defined(Py_DEBUG)
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
195 arena->total_allocs++;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
196 arena->total_size += size;
36728
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
197 #endif
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
198 /* Reset cur if we allocated a new block. */
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
199 if (arena->a_cur->ab_next) {
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
200 arena->a_cur = arena->a_cur->ab_next;
36728
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
201 #if defined(Py_DEBUG)
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
202 arena->total_blocks++;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
203 arena->total_block_size += arena->a_cur->ab_size;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
204 if (arena->a_cur->ab_size > DEFAULT_BLOCK_SIZE)
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
205 ++arena->total_big_blocks;
36728
f2a895d33a5c Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 36724
diff changeset
206 #endif
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
207 }
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
208 return p;
36065
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
209 }
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
210
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
211 int
36808
b1a3c2238f97 Trimmed trailing whitespace.
Tim Peters <tim.peters@gmail.com>
parents: 36765
diff changeset
212 PyArena_AddPyObject(PyArena *arena, PyObject *obj)
36065
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
213 {
61466
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
214 int r = PyList_Append(arena->a_objects, obj);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
215 if (r >= 0) {
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
216 Py_DECREF(obj);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
217 }
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents: 39442
diff changeset
218 return r;
36065
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff changeset
219 }