Mercurial > cpython-lru_cache-weakref
annotate Python/pyarena.c @ 73323:dad02a080bbc
Improve byte-compilation in packaging to be independent of -O or -B.
The code I fixed to comply with PEP 3147 still had one bug: When run
under python -O, some paths for pyc files would be pyo, because I called
imp.cache_from_source without explicit debug_override argument in some
places, and under -O that would return .pyo (this is well explained in
the imp docs). Now all code (util.byte_compile, build_py, install_lib)
can create .pyo files according to options given by users,
without interference from the calling Python’s own optimize mode.
On a related topic, I also removed the code that prevented byte
compilation under python -B. The rationale is that packaging gives
control over the creation of pyc files to the user with its own explicit
option, and the behavior should not be changed if the calling Python
happens to run with -B for whatever reason. I will argue that this is a
bug fix and ask to be allowed to backport this change to distutils.
Finally, I moved one nugget of information about the --compile and
--optimize options from the source into the doc. It clears up a
misunderstanding that I (and maybe other people) had.
| author | Éric Araujo <merwok@netwok.org> |
|---|---|
| date | Thu, 03 Nov 2011 05:08:28 +0100 |
| parents | 724920a015f7 |
| children | 99112b851b25 |
| rev | line source |
|---|---|
|
36065
ea083efdc96c
Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff
changeset
|
1 #include "Python.h" |
|
ea083efdc96c
Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff
changeset
|
2 |
|
36813
54754f0d25c7
Document the purpose of the struct _block members.
Tim Peters <tim.peters@gmail.com>
parents:
36812
diff
changeset
|
3 /* A simple arena block structure. |
|
36728
f2a895d33a5c
Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
36724
diff
changeset
|
4 |
|
f2a895d33a5c
Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
36724
diff
changeset
|
5 Measurements with standard library modules suggest the average |
|
f2a895d33a5c
Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
36724
diff
changeset
|
6 allocation is about 20 bytes and that most compiles use a single |
|
f2a895d33a5c
Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
36724
diff
changeset
|
7 block. |
|
37879
f731859e9b3b
Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
36813
diff
changeset
|
8 |
|
f731859e9b3b
Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
36813
diff
changeset
|
9 TODO(jhylton): Think about a realloc API, maybe just for the last |
|
f731859e9b3b
Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
36813
diff
changeset
|
10 allocation? |
|
36728
f2a895d33a5c
Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
36724
diff
changeset
|
11 */ |
|
36065
ea083efdc96c
Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff
changeset
|
12 |
|
36716
71d873054d29
Real arena implementation
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
36084
diff
changeset
|
13 #define DEFAULT_BLOCK_SIZE 8192 |
|
61466
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
14 #define ALIGNMENT 8 |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
15 #define ALIGNMENT_MASK (ALIGNMENT - 1) |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
16 #define ROUNDUP(x) (((x) + ALIGNMENT_MASK) & ~ALIGNMENT_MASK) |
|
37879
f731859e9b3b
Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
36813
diff
changeset
|
17 |
|
36716
71d873054d29
Real arena implementation
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
36084
diff
changeset
|
18 typedef struct _block { |
|
61466
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
19 /* Total number of bytes owned by this block available to pass out. |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
20 * Read-only after initialization. The first such byte starts at |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
21 * ab_mem. |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
22 */ |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
23 size_t ab_size; |
|
36813
54754f0d25c7
Document the purpose of the struct _block members.
Tim Peters <tim.peters@gmail.com>
parents:
36812
diff
changeset
|
24 |
|
61466
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
25 /* Total number of bytes already passed out. The next byte available |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
26 * to pass out starts at ab_mem + ab_offset. |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
27 */ |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
28 size_t ab_offset; |
|
36813
54754f0d25c7
Document the purpose of the struct _block members.
Tim Peters <tim.peters@gmail.com>
parents:
36812
diff
changeset
|
29 |
|
61466
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
30 /* An arena maintains a singly-linked, NULL-terminated list of |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
31 * all blocks owned by the arena. These are linked via the |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
32 * ab_next member. |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
33 */ |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
34 struct _block *ab_next; |
|
36813
54754f0d25c7
Document the purpose of the struct _block members.
Tim Peters <tim.peters@gmail.com>
parents:
36812
diff
changeset
|
35 |
|
61466
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
36 /* Pointer to the first allocatable byte owned by this block. Read- |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
37 * only after initialization. |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
38 */ |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
39 void *ab_mem; |
|
36716
71d873054d29
Real arena implementation
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
36084
diff
changeset
|
40 } block; |
|
36065
ea083efdc96c
Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff
changeset
|
41 |
|
36724
5f3822c2e731
Use simple PyList to implement list of PyObject pointers
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
36720
diff
changeset
|
42 /* The arena manages two kinds of memory, blocks of raw memory |
|
5f3822c2e731
Use simple PyList to implement list of PyObject pointers
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
36720
diff
changeset
|
43 and a list of PyObject* pointers. PyObjects are decrefed |
|
5f3822c2e731
Use simple PyList to implement list of PyObject pointers
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
36720
diff
changeset
|
44 when the arena is freed. |
|
5f3822c2e731
Use simple PyList to implement list of PyObject pointers
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
36720
diff
changeset
|
45 */ |
|
36808
b1a3c2238f97
Trimmed trailing whitespace.
Tim Peters <tim.peters@gmail.com>
parents:
36765
diff
changeset
|
46 |
|
36065
ea083efdc96c
Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff
changeset
|
47 struct _arena { |
|
61466
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
48 /* Pointer to the first block allocated for the arena, never NULL. |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
49 It is used only to find the first block when the arena is |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
50 being freed. |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
51 */ |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
52 block *a_head; |
|
37879
f731859e9b3b
Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
36813
diff
changeset
|
53 |
|
61466
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
54 /* Pointer to the block currently used for allocation. It's |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
55 ab_next field should be NULL. If it is not-null after a |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
56 call to block_alloc(), it means a new block has been allocated |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
57 and a_cur should be reset to point it. |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
58 */ |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
59 block *a_cur; |
|
37879
f731859e9b3b
Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
36813
diff
changeset
|
60 |
|
61466
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
61 /* A Python list object containing references to all the PyObject |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
62 pointers associated with this area. They will be DECREFed |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
63 when the arena is freed. |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
64 */ |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
65 PyObject *a_objects; |
|
37879
f731859e9b3b
Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
36813
diff
changeset
|
66 |
|
36728
f2a895d33a5c
Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
36724
diff
changeset
|
67 #if defined(Py_DEBUG) |
|
61466
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
68 /* Debug output */ |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
69 size_t total_allocs; |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
70 size_t total_size; |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
71 size_t total_blocks; |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
72 size_t total_block_size; |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
73 size_t total_big_blocks; |
|
36728
f2a895d33a5c
Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
36724
diff
changeset
|
74 #endif |
|
36065
ea083efdc96c
Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff
changeset
|
75 }; |
|
ea083efdc96c
Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff
changeset
|
76 |
|
36716
71d873054d29
Real arena implementation
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
36084
diff
changeset
|
77 static block * |
|
71d873054d29
Real arena implementation
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
36084
diff
changeset
|
78 block_new(size_t size) |
|
71d873054d29
Real arena implementation
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
36084
diff
changeset
|
79 { |
|
61466
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
80 /* Allocate header and block as one unit. |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
81 ab_mem points just past header. */ |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
82 block *b = (block *)malloc(sizeof(block) + size); |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
83 if (!b) |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
84 return NULL; |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
85 b->ab_size = size; |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
86 b->ab_mem = (void *)(b + 1); |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
87 b->ab_next = NULL; |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
88 b->ab_offset = ROUNDUP((Py_uintptr_t)(b->ab_mem)) - |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
89 (Py_uintptr_t)(b->ab_mem); |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
90 return b; |
|
36065
ea083efdc96c
Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff
changeset
|
91 } |
|
ea083efdc96c
Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff
changeset
|
92 |
|
ea083efdc96c
Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff
changeset
|
93 static void |
|
36716
71d873054d29
Real arena implementation
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
36084
diff
changeset
|
94 block_free(block *b) { |
|
61466
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
95 while (b) { |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
96 block *next = b->ab_next; |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
97 free(b); |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
98 b = next; |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
99 } |
|
36065
ea083efdc96c
Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff
changeset
|
100 } |
|
ea083efdc96c
Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff
changeset
|
101 |
|
36716
71d873054d29
Real arena implementation
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
36084
diff
changeset
|
102 static void * |
|
71d873054d29
Real arena implementation
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
36084
diff
changeset
|
103 block_alloc(block *b, size_t size) |
|
71d873054d29
Real arena implementation
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
36084
diff
changeset
|
104 { |
|
61466
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
105 void *p; |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
106 assert(b); |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
107 size = ROUNDUP(size); |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
108 if (b->ab_offset + size > b->ab_size) { |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
109 /* If we need to allocate more memory than will fit in |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
110 the default block, allocate a one-off block that is |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
111 exactly the right size. */ |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
112 /* TODO(jhylton): Think about space waste at end of block */ |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
113 block *newbl = block_new( |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
114 size < DEFAULT_BLOCK_SIZE ? |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
115 DEFAULT_BLOCK_SIZE : size); |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
116 if (!newbl) |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
117 return NULL; |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
118 assert(!b->ab_next); |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
119 b->ab_next = newbl; |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
120 b = newbl; |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
121 } |
|
36716
71d873054d29
Real arena implementation
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
36084
diff
changeset
|
122 |
|
61466
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
123 assert(b->ab_offset + size <= b->ab_size); |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
124 p = (void *)(((char *)b->ab_mem) + b->ab_offset); |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
125 b->ab_offset += size; |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
126 return p; |
|
36716
71d873054d29
Real arena implementation
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
36084
diff
changeset
|
127 } |
|
36065
ea083efdc96c
Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff
changeset
|
128 |
|
ea083efdc96c
Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff
changeset
|
129 PyArena * |
|
ea083efdc96c
Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff
changeset
|
130 PyArena_New() |
|
ea083efdc96c
Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff
changeset
|
131 { |
|
61466
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
132 PyArena* arena = (PyArena *)malloc(sizeof(PyArena)); |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
133 if (!arena) |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
134 return (PyArena*)PyErr_NoMemory(); |
|
36065
ea083efdc96c
Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff
changeset
|
135 |
|
61466
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
136 arena->a_head = block_new(DEFAULT_BLOCK_SIZE); |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
137 arena->a_cur = arena->a_head; |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
138 if (!arena->a_head) { |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
139 free((void *)arena); |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
140 return (PyArena*)PyErr_NoMemory(); |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
141 } |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
142 arena->a_objects = PyList_New(0); |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
143 if (!arena->a_objects) { |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
144 block_free(arena->a_head); |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
145 free((void *)arena); |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
146 return (PyArena*)PyErr_NoMemory(); |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
147 } |
|
36728
f2a895d33a5c
Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
36724
diff
changeset
|
148 #if defined(Py_DEBUG) |
|
61466
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
149 arena->total_allocs = 0; |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
150 arena->total_size = 0; |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
151 arena->total_blocks = 1; |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
152 arena->total_block_size = DEFAULT_BLOCK_SIZE; |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
153 arena->total_big_blocks = 0; |
|
36728
f2a895d33a5c
Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
36724
diff
changeset
|
154 #endif |
|
61466
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
155 return arena; |
|
36065
ea083efdc96c
Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff
changeset
|
156 } |
|
ea083efdc96c
Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff
changeset
|
157 |
|
ea083efdc96c
Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff
changeset
|
158 void |
|
ea083efdc96c
Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff
changeset
|
159 PyArena_Free(PyArena *arena) |
|
ea083efdc96c
Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff
changeset
|
160 { |
|
61466
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
161 int r; |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
162 assert(arena); |
|
36728
f2a895d33a5c
Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
36724
diff
changeset
|
163 #if defined(Py_DEBUG) |
|
61466
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
164 /* |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
165 fprintf(stderr, |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
166 "alloc=%d size=%d blocks=%d block_size=%d big=%d objects=%d\n", |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
167 arena->total_allocs, arena->total_size, arena->total_blocks, |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
168 arena->total_block_size, arena->total_big_blocks, |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
169 PyList_Size(arena->a_objects)); |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
170 */ |
|
36728
f2a895d33a5c
Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
36724
diff
changeset
|
171 #endif |
|
61466
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
172 block_free(arena->a_head); |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
173 /* This property normally holds, except when the code being compiled |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
174 is sys.getobjects(0), in which case there will be two references. |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
175 assert(arena->a_objects->ob_refcnt == 1); |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
176 */ |
|
37879
f731859e9b3b
Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents:
36813
diff
changeset
|
177 |
|
61466
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
178 /* Clear all the elements from the list. This is necessary |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
179 to guarantee that they will be DECREFed. */ |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
180 r = PyList_SetSlice(arena->a_objects, |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
181 0, PyList_GET_SIZE(arena->a_objects), NULL); |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
182 assert(r == 0); |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
183 assert(PyList_GET_SIZE(arena->a_objects) == 0); |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
184 Py_DECREF(arena->a_objects); |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
185 free(arena); |
|
36065
ea083efdc96c
Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff
changeset
|
186 } |
|
ea083efdc96c
Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff
changeset
|
187 |
|
ea083efdc96c
Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff
changeset
|
188 void * |
|
36808
b1a3c2238f97
Trimmed trailing whitespace.
Tim Peters <tim.peters@gmail.com>
parents:
36765
diff
changeset
|
189 PyArena_Malloc(PyArena *arena, size_t size) |
|
36065
ea083efdc96c
Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff
changeset
|
190 { |
|
61466
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
191 void *p = block_alloc(arena->a_cur, size); |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
192 if (!p) |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
193 return PyErr_NoMemory(); |
|
36728
f2a895d33a5c
Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
36724
diff
changeset
|
194 #if defined(Py_DEBUG) |
|
61466
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
195 arena->total_allocs++; |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
196 arena->total_size += size; |
|
36728
f2a895d33a5c
Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
36724
diff
changeset
|
197 #endif |
|
61466
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
198 /* Reset cur if we allocated a new block. */ |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
199 if (arena->a_cur->ab_next) { |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
200 arena->a_cur = arena->a_cur->ab_next; |
|
36728
f2a895d33a5c
Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
36724
diff
changeset
|
201 #if defined(Py_DEBUG) |
|
61466
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
202 arena->total_blocks++; |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
203 arena->total_block_size += arena->a_cur->ab_size; |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
204 if (arena->a_cur->ab_size > DEFAULT_BLOCK_SIZE) |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
205 ++arena->total_big_blocks; |
|
36728
f2a895d33a5c
Add some stats collection in debugging mode.
Jeremy Hylton <jeremy@alum.mit.edu>
parents:
36724
diff
changeset
|
206 #endif |
|
61466
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
207 } |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
208 return p; |
|
36065
ea083efdc96c
Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff
changeset
|
209 } |
|
ea083efdc96c
Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff
changeset
|
210 |
|
ea083efdc96c
Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff
changeset
|
211 int |
|
36808
b1a3c2238f97
Trimmed trailing whitespace.
Tim Peters <tim.peters@gmail.com>
parents:
36765
diff
changeset
|
212 PyArena_AddPyObject(PyArena *arena, PyObject *obj) |
|
36065
ea083efdc96c
Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff
changeset
|
213 { |
|
61466
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
214 int r = PyList_Append(arena->a_objects, obj); |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
215 if (r >= 0) { |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
216 Py_DECREF(obj); |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
217 } |
|
cab14be0ada1
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
39442
diff
changeset
|
218 return r; |
|
36065
ea083efdc96c
Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines,
Neal Norwitz <nnorwitz@gmail.com>
parents:
diff
changeset
|
219 } |
