cpython-lru_cache-weakref: Python/pyarena.c annotate

annotate Python/pyarena.c @ 87690:66c1c9f32567 default tip

Add use_weakref option to functools.lru_cache The use_weakref option makes lru_cache create weak references to the function arguments when creating the cache key, which prevents them from staying in the memory forever even after all other references to them are gone. This is a proof of concept for http://bugs.python.org/issue19859

author	Radomir Dopieralski <devel@sheep.art.pl>
date	Tue, 03 Dec 2013 16:02:48 +0100
parents	fb7d346b45fa
children

rev	line source
36065 ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines, Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1 #include "Python.h"
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines, Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2
36813 54754f0d25c7 Document the purpose of the struct _block members. Tim Peters <tim.peters@gmail.com> parents: 36812 diff changeset	3 /* A simple arena block structure.
36728 f2a895d33a5c Add some stats collection in debugging mode. Jeremy Hylton <jeremy@alum.mit.edu> parents: 36724 diff changeset	4
f2a895d33a5c Add some stats collection in debugging mode. Jeremy Hylton <jeremy@alum.mit.edu> parents: 36724 diff changeset	5 Measurements with standard library modules suggest the average
f2a895d33a5c Add some stats collection in debugging mode. Jeremy Hylton <jeremy@alum.mit.edu> parents: 36724 diff changeset	6 allocation is about 20 bytes and that most compiles use a single
f2a895d33a5c Add some stats collection in debugging mode. Jeremy Hylton <jeremy@alum.mit.edu> parents: 36724 diff changeset	7 block.
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 36813 diff changeset	8
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 36813 diff changeset	9 TODO(jhylton): Think about a realloc API, maybe just for the last
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 36813 diff changeset	10 allocation?
36728 f2a895d33a5c Add some stats collection in debugging mode. Jeremy Hylton <jeremy@alum.mit.edu> parents: 36724 diff changeset	11 */
36065 ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines, Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	12
36716 71d873054d29 Real arena implementation Jeremy Hylton <jeremy@alum.mit.edu> parents: 36084 diff changeset	13 #define DEFAULT_BLOCK_SIZE 8192
61466 cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	14 #define ALIGNMENT 8
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 36813 diff changeset	15
36716 71d873054d29 Real arena implementation Jeremy Hylton <jeremy@alum.mit.edu> parents: 36084 diff changeset	16 typedef struct _block {
61466 cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	17 /* Total number of bytes owned by this block available to pass out.
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	18 * Read-only after initialization. The first such byte starts at
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	19 * ab_mem.
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	20 */
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	21 size_t ab_size;
36813 54754f0d25c7 Document the purpose of the struct _block members. Tim Peters <tim.peters@gmail.com> parents: 36812 diff changeset	22
61466 cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	23 /* Total number of bytes already passed out. The next byte available
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	24 * to pass out starts at ab_mem + ab_offset.
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	25 */
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	26 size_t ab_offset;
36813 54754f0d25c7 Document the purpose of the struct _block members. Tim Peters <tim.peters@gmail.com> parents: 36812 diff changeset	27
61466 cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	28 /* An arena maintains a singly-linked, NULL-terminated list of
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	29 * all blocks owned by the arena. These are linked via the
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	30 * ab_next member.
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	31 */
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	32 struct _block *ab_next;
36813 54754f0d25c7 Document the purpose of the struct _block members. Tim Peters <tim.peters@gmail.com> parents: 36812 diff changeset	33
61466 cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	34 /* Pointer to the first allocatable byte owned by this block. Read-
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	35 * only after initialization.
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	36 */
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	37 void *ab_mem;
36716 71d873054d29 Real arena implementation Jeremy Hylton <jeremy@alum.mit.edu> parents: 36084 diff changeset	38 } block;
36065 ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines, Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	39
36724 5f3822c2e731 Use simple PyList to implement list of PyObject pointers Jeremy Hylton <jeremy@alum.mit.edu> parents: 36720 diff changeset	40 /* The arena manages two kinds of memory, blocks of raw memory
5f3822c2e731 Use simple PyList to implement list of PyObject pointers Jeremy Hylton <jeremy@alum.mit.edu> parents: 36720 diff changeset	41 and a list of PyObject* pointers. PyObjects are decrefed
5f3822c2e731 Use simple PyList to implement list of PyObject pointers Jeremy Hylton <jeremy@alum.mit.edu> parents: 36720 diff changeset	42 when the arena is freed.
5f3822c2e731 Use simple PyList to implement list of PyObject pointers Jeremy Hylton <jeremy@alum.mit.edu> parents: 36720 diff changeset	43 */
36808 b1a3c2238f97 Trimmed trailing whitespace. Tim Peters <tim.peters@gmail.com> parents: 36765 diff changeset	44
36065 ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines, Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	45 struct _arena {
61466 cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	46 /* Pointer to the first block allocated for the arena, never NULL.
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	47 It is used only to find the first block when the arena is
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	48 being freed.
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	49 */
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	50 block *a_head;
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 36813 diff changeset	51
61466 cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	52 /* Pointer to the block currently used for allocation. It's
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	53 ab_next field should be NULL. If it is not-null after a
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	54 call to block_alloc(), it means a new block has been allocated
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	55 and a_cur should be reset to point it.
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	56 */
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	57 block *a_cur;
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 36813 diff changeset	58
61466 cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	59 /* A Python list object containing references to all the PyObject
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	60 pointers associated with this area. They will be DECREFed
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	61 when the arena is freed.
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	62 */
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	63 PyObject *a_objects;
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 36813 diff changeset	64
36728 f2a895d33a5c Add some stats collection in debugging mode. Jeremy Hylton <jeremy@alum.mit.edu> parents: 36724 diff changeset	65 #if defined(Py_DEBUG)
61466 cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	66 /* Debug output */
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	67 size_t total_allocs;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	68 size_t total_size;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	69 size_t total_blocks;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	70 size_t total_block_size;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	71 size_t total_big_blocks;
36728 f2a895d33a5c Add some stats collection in debugging mode. Jeremy Hylton <jeremy@alum.mit.edu> parents: 36724 diff changeset	72 #endif
36065 ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines, Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	73 };
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines, Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	74
36716 71d873054d29 Real arena implementation Jeremy Hylton <jeremy@alum.mit.edu> parents: 36084 diff changeset	75 static block *
71d873054d29 Real arena implementation Jeremy Hylton <jeremy@alum.mit.edu> parents: 36084 diff changeset	76 block_new(size_t size)
71d873054d29 Real arena implementation Jeremy Hylton <jeremy@alum.mit.edu> parents: 36084 diff changeset	77 {
61466 cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	78 /* Allocate header and block as one unit.
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	79 ab_mem points just past header. */
84498 fb7d346b45fa Issue #18203: Replace malloc() with PyMem_Malloc() to allocate arena objects Victor Stinner <victor.stinner@gmail.com> parents: 81602 diff changeset	80 block b = (block )PyMem_Malloc(sizeof(block) + size);
61466 cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	81 if (!b)
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	82 return NULL;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	83 b->ab_size = size;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	84 b->ab_mem = (void *)(b + 1);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	85 b->ab_next = NULL;
79065 99112b851b25 Issue #15144: Fix possible integer overflow when handling pointers as integer values, by using Py_uintptr_t instead of size_t. Antoine Pitrou <solipsis@pitrou.net> parents: 66556 diff changeset	86 b->ab_offset = (char *)_Py_ALIGN_UP(b->ab_mem, ALIGNMENT) -
99112b851b25 Issue #15144: Fix possible integer overflow when handling pointers as integer values, by using Py_uintptr_t instead of size_t. Antoine Pitrou <solipsis@pitrou.net> parents: 66556 diff changeset	87 (char *)(b->ab_mem);
61466 cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	88 return b;
36065 ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines, Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	89 }
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines, Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	90
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines, Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	91 static void
36716 71d873054d29 Real arena implementation Jeremy Hylton <jeremy@alum.mit.edu> parents: 36084 diff changeset	92 block_free(block *b) {
61466 cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	93 while (b) {
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	94 block *next = b->ab_next;
84498 fb7d346b45fa Issue #18203: Replace malloc() with PyMem_Malloc() to allocate arena objects Victor Stinner <victor.stinner@gmail.com> parents: 81602 diff changeset	95 PyMem_Free(b);
61466 cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	96 b = next;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	97 }
36065 ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines, Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	98 }
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines, Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	99
36716 71d873054d29 Real arena implementation Jeremy Hylton <jeremy@alum.mit.edu> parents: 36084 diff changeset	100 static void *
71d873054d29 Real arena implementation Jeremy Hylton <jeremy@alum.mit.edu> parents: 36084 diff changeset	101 block_alloc(block *b, size_t size)
71d873054d29 Real arena implementation Jeremy Hylton <jeremy@alum.mit.edu> parents: 36084 diff changeset	102 {
61466 cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	103 void *p;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	104 assert(b);
79065 99112b851b25 Issue #15144: Fix possible integer overflow when handling pointers as integer values, by using Py_uintptr_t instead of size_t. Antoine Pitrou <solipsis@pitrou.net> parents: 66556 diff changeset	105 size = _Py_SIZE_ROUND_UP(size, ALIGNMENT);
61466 cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	106 if (b->ab_offset + size > b->ab_size) {
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	107 /* If we need to allocate more memory than will fit in
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	108 the default block, allocate a one-off block that is
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	109 exactly the right size. */
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	110 /* TODO(jhylton): Think about space waste at end of block */
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	111 block *newbl = block_new(
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	112 size < DEFAULT_BLOCK_SIZE ?
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	113 DEFAULT_BLOCK_SIZE : size);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	114 if (!newbl)
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	115 return NULL;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	116 assert(!b->ab_next);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	117 b->ab_next = newbl;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	118 b = newbl;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	119 }
36716 71d873054d29 Real arena implementation Jeremy Hylton <jeremy@alum.mit.edu> parents: 36084 diff changeset	120
61466 cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	121 assert(b->ab_offset + size <= b->ab_size);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	122 p = (void )(((char )b->ab_mem) + b->ab_offset);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	123 b->ab_offset += size;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	124 return p;
36716 71d873054d29 Real arena implementation Jeremy Hylton <jeremy@alum.mit.edu> parents: 36084 diff changeset	125 }
36065 ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines, Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	126
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines, Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	127 PyArena *
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines, Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	128 PyArena_New()
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines, Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	129 {
84498 fb7d346b45fa Issue #18203: Replace malloc() with PyMem_Malloc() to allocate arena objects Victor Stinner <victor.stinner@gmail.com> parents: 81602 diff changeset	130 PyArena* arena = (PyArena *)PyMem_Malloc(sizeof(PyArena));
61466 cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	131 if (!arena)
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	132 return (PyArena*)PyErr_NoMemory();
36065 ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines, Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	133
61466 cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	134 arena->a_head = block_new(DEFAULT_BLOCK_SIZE);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	135 arena->a_cur = arena->a_head;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	136 if (!arena->a_head) {
84498 fb7d346b45fa Issue #18203: Replace malloc() with PyMem_Malloc() to allocate arena objects Victor Stinner <victor.stinner@gmail.com> parents: 81602 diff changeset	137 PyMem_Free((void *)arena);
61466 cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	138 return (PyArena*)PyErr_NoMemory();
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	139 }
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	140 arena->a_objects = PyList_New(0);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	141 if (!arena->a_objects) {
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	142 block_free(arena->a_head);
84498 fb7d346b45fa Issue #18203: Replace malloc() with PyMem_Malloc() to allocate arena objects Victor Stinner <victor.stinner@gmail.com> parents: 81602 diff changeset	143 PyMem_Free((void *)arena);
61466 cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	144 return (PyArena*)PyErr_NoMemory();
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	145 }
36728 f2a895d33a5c Add some stats collection in debugging mode. Jeremy Hylton <jeremy@alum.mit.edu> parents: 36724 diff changeset	146 #if defined(Py_DEBUG)
61466 cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	147 arena->total_allocs = 0;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	148 arena->total_size = 0;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	149 arena->total_blocks = 1;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	150 arena->total_block_size = DEFAULT_BLOCK_SIZE;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	151 arena->total_big_blocks = 0;
36728 f2a895d33a5c Add some stats collection in debugging mode. Jeremy Hylton <jeremy@alum.mit.edu> parents: 36724 diff changeset	152 #endif
61466 cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	153 return arena;
36065 ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines, Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	154 }
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines, Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	155
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines, Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	156 void
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines, Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	157 PyArena_Free(PyArena *arena)
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines, Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	158 {
61466 cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	159 assert(arena);
36728 f2a895d33a5c Add some stats collection in debugging mode. Jeremy Hylton <jeremy@alum.mit.edu> parents: 36724 diff changeset	160 #if defined(Py_DEBUG)
61466 cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	161 /*
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	162 fprintf(stderr,
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	163 "alloc=%d size=%d blocks=%d block_size=%d big=%d objects=%d\n",
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	164 arena->total_allocs, arena->total_size, arena->total_blocks,
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	165 arena->total_block_size, arena->total_big_blocks,
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	166 PyList_Size(arena->a_objects));
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	167 */
36728 f2a895d33a5c Add some stats collection in debugging mode. Jeremy Hylton <jeremy@alum.mit.edu> parents: 36724 diff changeset	168 #endif
61466 cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	169 block_free(arena->a_head);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	170 /* This property normally holds, except when the code being compiled
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	171 is sys.getobjects(0), in which case there will be two references.
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	172 assert(arena->a_objects->ob_refcnt == 1);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	173 */
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 36813 diff changeset	174
61466 cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	175 Py_DECREF(arena->a_objects);
84498 fb7d346b45fa Issue #18203: Replace malloc() with PyMem_Malloc() to allocate arena objects Victor Stinner <victor.stinner@gmail.com> parents: 81602 diff changeset	176 PyMem_Free(arena);
36065 ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines, Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	177 }
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines, Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	178
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines, Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	179 void *
36808 b1a3c2238f97 Trimmed trailing whitespace. Tim Peters <tim.peters@gmail.com> parents: 36765 diff changeset	180 PyArena_Malloc(PyArena *arena, size_t size)
36065 ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines, Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	181 {
61466 cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	182 void *p = block_alloc(arena->a_cur, size);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	183 if (!p)
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	184 return PyErr_NoMemory();
36728 f2a895d33a5c Add some stats collection in debugging mode. Jeremy Hylton <jeremy@alum.mit.edu> parents: 36724 diff changeset	185 #if defined(Py_DEBUG)
61466 cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	186 arena->total_allocs++;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	187 arena->total_size += size;
36728 f2a895d33a5c Add some stats collection in debugging mode. Jeremy Hylton <jeremy@alum.mit.edu> parents: 36724 diff changeset	188 #endif
61466 cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	189 /* Reset cur if we allocated a new block. */
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	190 if (arena->a_cur->ab_next) {
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	191 arena->a_cur = arena->a_cur->ab_next;
36728 f2a895d33a5c Add some stats collection in debugging mode. Jeremy Hylton <jeremy@alum.mit.edu> parents: 36724 diff changeset	192 #if defined(Py_DEBUG)
61466 cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	193 arena->total_blocks++;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	194 arena->total_block_size += arena->a_cur->ab_size;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	195 if (arena->a_cur->ab_size > DEFAULT_BLOCK_SIZE)
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	196 ++arena->total_big_blocks;
36728 f2a895d33a5c Add some stats collection in debugging mode. Jeremy Hylton <jeremy@alum.mit.edu> parents: 36724 diff changeset	197 #endif
61466 cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	198 }
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	199 return p;
36065 ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines, Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	200 }
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines, Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	201
ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines, Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	202 int
36808 b1a3c2238f97 Trimmed trailing whitespace. Tim Peters <tim.peters@gmail.com> parents: 36765 diff changeset	203 PyArena_AddPyObject(PyArena arena, PyObject obj)
36065 ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines, Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	204 {
61466 cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	205 int r = PyList_Append(arena->a_objects, obj);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	206 if (r >= 0) {
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	207 Py_DECREF(obj);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	208 }
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 39442 diff changeset	209 return r;
36065 ea083efdc96c Merge from ast-arena. This reduces the code in Python/ast.c by ~300 lines, Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	210 }

Mercurial > cpython-lru_cache-weakref

annotate Python/pyarena.c @ 87690:66c1c9f32567 default tip