cpython-lru_cache-weakref: Python/mysnprintf.c annotate

annotate Python/mysnprintf.c @ 87690:66c1c9f32567 default tip

Add use_weakref option to functools.lru_cache The use_weakref option makes lru_cache create weak references to the function arguments when creating the cache key, which prevents them from staying in the memory forever even after all other references to them are gone. This is a proof of concept for http://bugs.python.org/issue19859

author	Radomir Dopieralski <devel@sheep.art.pl>
date	Tue, 03 Dec 2013 16:02:48 +0100
parents	cab14be0ada1
children

rev	line source
18908 79c676f9b34a This patch turns the Python API mismatch notice into a standard Marc-André Lemburg <mal@egenix.com> parents: diff changeset	1 #include "Python.h"
79c676f9b34a This patch turns the Python API mismatch notice into a standard Marc-André Lemburg <mal@egenix.com> parents: diff changeset	2
21145 87fa6e9bd77e mysnprintf.c: Massive rewrite of PyOS_snprintf and PyOS_vsnprintf, to Tim Peters <tim.peters@gmail.com> parents: 21126 diff changeset	3 /* snprintf() wrappers. If the platform has vsnprintf, we use it, else we
87fa6e9bd77e mysnprintf.c: Massive rewrite of PyOS_snprintf and PyOS_vsnprintf, to Tim Peters <tim.peters@gmail.com> parents: 21126 diff changeset	4 emulate it in a half-hearted way. Even if the platform has it, we wrap
87fa6e9bd77e mysnprintf.c: Massive rewrite of PyOS_snprintf and PyOS_vsnprintf, to Tim Peters <tim.peters@gmail.com> parents: 21126 diff changeset	5 it because platforms differ in what vsnprintf does in case the buffer
87fa6e9bd77e mysnprintf.c: Massive rewrite of PyOS_snprintf and PyOS_vsnprintf, to Tim Peters <tim.peters@gmail.com> parents: 21126 diff changeset	6 is too small: C99 behavior is to return the number of characters that
87fa6e9bd77e mysnprintf.c: Massive rewrite of PyOS_snprintf and PyOS_vsnprintf, to Tim Peters <tim.peters@gmail.com> parents: 21126 diff changeset	7 would have been written had the buffer not been too small, and to set
87fa6e9bd77e mysnprintf.c: Massive rewrite of PyOS_snprintf and PyOS_vsnprintf, to Tim Peters <tim.peters@gmail.com> parents: 21126 diff changeset	8 the last byte of the buffer to \0. At least MS _vsnprintf returns a
87fa6e9bd77e mysnprintf.c: Massive rewrite of PyOS_snprintf and PyOS_vsnprintf, to Tim Peters <tim.peters@gmail.com> parents: 21126 diff changeset	9 negative value instead, and fills the entire buffer with non-\0 data.
87fa6e9bd77e mysnprintf.c: Massive rewrite of PyOS_snprintf and PyOS_vsnprintf, to Tim Peters <tim.peters@gmail.com> parents: 21126 diff changeset	10
87fa6e9bd77e mysnprintf.c: Massive rewrite of PyOS_snprintf and PyOS_vsnprintf, to Tim Peters <tim.peters@gmail.com> parents: 21126 diff changeset	11 The wrappers ensure that str[size-1] is always \0 upon return.
87fa6e9bd77e mysnprintf.c: Massive rewrite of PyOS_snprintf and PyOS_vsnprintf, to Tim Peters <tim.peters@gmail.com> parents: 21126 diff changeset	12
87fa6e9bd77e mysnprintf.c: Massive rewrite of PyOS_snprintf and PyOS_vsnprintf, to Tim Peters <tim.peters@gmail.com> parents: 21126 diff changeset	13 PyOS_snprintf and PyOS_vsnprintf never write more than size bytes
87fa6e9bd77e mysnprintf.c: Massive rewrite of PyOS_snprintf and PyOS_vsnprintf, to Tim Peters <tim.peters@gmail.com> parents: 21126 diff changeset	14 (including the trailing '\0') into str.
87fa6e9bd77e mysnprintf.c: Massive rewrite of PyOS_snprintf and PyOS_vsnprintf, to Tim Peters <tim.peters@gmail.com> parents: 21126 diff changeset	15
87fa6e9bd77e mysnprintf.c: Massive rewrite of PyOS_snprintf and PyOS_vsnprintf, to Tim Peters <tim.peters@gmail.com> parents: 21126 diff changeset	16 If the platform doesn't have vsnprintf, and the buffer size needed to
87fa6e9bd77e mysnprintf.c: Massive rewrite of PyOS_snprintf and PyOS_vsnprintf, to Tim Peters <tim.peters@gmail.com> parents: 21126 diff changeset	17 avoid truncation exceeds size by more than 512, Python aborts with a
87fa6e9bd77e mysnprintf.c: Massive rewrite of PyOS_snprintf and PyOS_vsnprintf, to Tim Peters <tim.peters@gmail.com> parents: 21126 diff changeset	18 Py_FatalError.
87fa6e9bd77e mysnprintf.c: Massive rewrite of PyOS_snprintf and PyOS_vsnprintf, to Tim Peters <tim.peters@gmail.com> parents: 21126 diff changeset	19
87fa6e9bd77e mysnprintf.c: Massive rewrite of PyOS_snprintf and PyOS_vsnprintf, to Tim Peters <tim.peters@gmail.com> parents: 21126 diff changeset	20 Return value (rv):
18908 79c676f9b34a This patch turns the Python API mismatch notice into a standard Marc-André Lemburg <mal@egenix.com> parents: diff changeset	21
61466 cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52976 diff changeset	22 When 0 <= rv < size, the output conversion was unexceptional, and
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52976 diff changeset	23 rv characters were written to str (excluding a trailing \0 byte at
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52976 diff changeset	24 str[rv]).
21145 87fa6e9bd77e mysnprintf.c: Massive rewrite of PyOS_snprintf and PyOS_vsnprintf, to Tim Peters <tim.peters@gmail.com> parents: 21126 diff changeset	25
61466 cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52976 diff changeset	26 When rv >= size, output conversion was truncated, and a buffer of
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52976 diff changeset	27 size rv+1 would have been needed to avoid truncation. str[size-1]
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52976 diff changeset	28 is \0 in this case.
21145 87fa6e9bd77e mysnprintf.c: Massive rewrite of PyOS_snprintf and PyOS_vsnprintf, to Tim Peters <tim.peters@gmail.com> parents: 21126 diff changeset	29
61466 cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52976 diff changeset	30 When rv < 0, "something bad happened". str[size-1] is \0 in this
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52976 diff changeset	31 case too, but the rest of str is unreliable. It could be that
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52976 diff changeset	32 an error in format codes was detected by libc, or on platforms
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52976 diff changeset	33 with a non-C99 vsnprintf simply that the buffer wasn't big enough
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52976 diff changeset	34 to avoid truncation, or on platforms without any vsnprintf that
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52976 diff changeset	35 PyMem_Malloc couldn't obtain space for a temp buffer.
21145 87fa6e9bd77e mysnprintf.c: Massive rewrite of PyOS_snprintf and PyOS_vsnprintf, to Tim Peters <tim.peters@gmail.com> parents: 21126 diff changeset	36
87fa6e9bd77e mysnprintf.c: Massive rewrite of PyOS_snprintf and PyOS_vsnprintf, to Tim Peters <tim.peters@gmail.com> parents: 21126 diff changeset	37 CAUTION: Unlike C99, str != NULL and size > 0 are required.
87fa6e9bd77e mysnprintf.c: Massive rewrite of PyOS_snprintf and PyOS_vsnprintf, to Tim Peters <tim.peters@gmail.com> parents: 21126 diff changeset	38 */
87fa6e9bd77e mysnprintf.c: Massive rewrite of PyOS_snprintf and PyOS_vsnprintf, to Tim Peters <tim.peters@gmail.com> parents: 21126 diff changeset	39
87fa6e9bd77e mysnprintf.c: Massive rewrite of PyOS_snprintf and PyOS_vsnprintf, to Tim Peters <tim.peters@gmail.com> parents: 21126 diff changeset	40 int
87fa6e9bd77e mysnprintf.c: Massive rewrite of PyOS_snprintf and PyOS_vsnprintf, to Tim Peters <tim.peters@gmail.com> parents: 21126 diff changeset	41 PyOS_snprintf(char str, size_t size, const char format, ...)
18908 79c676f9b34a This patch turns the Python API mismatch notice into a standard Marc-André Lemburg <mal@egenix.com> parents: diff changeset	42 {
61466 cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52976 diff changeset	43 int rc;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52976 diff changeset	44 va_list va;
21145 87fa6e9bd77e mysnprintf.c: Massive rewrite of PyOS_snprintf and PyOS_vsnprintf, to Tim Peters <tim.peters@gmail.com> parents: 21126 diff changeset	45
61466 cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52976 diff changeset	46 va_start(va, format);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52976 diff changeset	47 rc = PyOS_vsnprintf(str, size, format, va);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52976 diff changeset	48 va_end(va);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52976 diff changeset	49 return rc;
18908 79c676f9b34a This patch turns the Python API mismatch notice into a standard Marc-André Lemburg <mal@egenix.com> parents: diff changeset	50 }
79c676f9b34a This patch turns the Python API mismatch notice into a standard Marc-André Lemburg <mal@egenix.com> parents: diff changeset	51
21145 87fa6e9bd77e mysnprintf.c: Massive rewrite of PyOS_snprintf and PyOS_vsnprintf, to Tim Peters <tim.peters@gmail.com> parents: 21126 diff changeset	52 int
87fa6e9bd77e mysnprintf.c: Massive rewrite of PyOS_snprintf and PyOS_vsnprintf, to Tim Peters <tim.peters@gmail.com> parents: 21126 diff changeset	53 PyOS_vsnprintf(char str, size_t size, const char format, va_list va)
18908 79c676f9b34a This patch turns the Python API mismatch notice into a standard Marc-André Lemburg <mal@egenix.com> parents: diff changeset	54 {
61466 cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52976 diff changeset	55 int len; /* # bytes written, excluding \0 */
47559 2925b568aacc Merged revisions 63562,63570,63728,63734,63784,63788,63802,63817,63827,63839,63887,63975,63998 via svnmerge from Georg Brandl <georg@python.org> parents: 21518 diff changeset	56 #ifdef HAVE_SNPRINTF
2925b568aacc Merged revisions 63562,63570,63728,63734,63784,63788,63802,63817,63827,63839,63887,63975,63998 via svnmerge from Georg Brandl <georg@python.org> parents: 21518 diff changeset	57 #define _PyOS_vsnprintf_EXTRA_SPACE 1
2925b568aacc Merged revisions 63562,63570,63728,63734,63784,63788,63802,63817,63827,63839,63887,63975,63998 via svnmerge from Georg Brandl <georg@python.org> parents: 21518 diff changeset	58 #else
2925b568aacc Merged revisions 63562,63570,63728,63734,63784,63788,63802,63817,63827,63839,63887,63975,63998 via svnmerge from Georg Brandl <georg@python.org> parents: 21518 diff changeset	59 #define _PyOS_vsnprintf_EXTRA_SPACE 512
61466 cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52976 diff changeset	60 char *buffer;
18908 79c676f9b34a This patch turns the Python API mismatch notice into a standard Marc-André Lemburg <mal@egenix.com> parents: diff changeset	61 #endif
61466 cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52976 diff changeset	62 assert(str != NULL);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52976 diff changeset	63 assert(size > 0);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52976 diff changeset	64 assert(format != NULL);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52976 diff changeset	65 /* We take a size_t as input but return an int. Sanity check
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52976 diff changeset	66 * our input so that it won't cause an overflow in the
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52976 diff changeset	67 * vsnprintf return value or the buffer malloc size. */
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52976 diff changeset	68 if (size > INT_MAX - _PyOS_vsnprintf_EXTRA_SPACE) {
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52976 diff changeset	69 len = -666;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52976 diff changeset	70 goto Done;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52976 diff changeset	71 }
18908 79c676f9b34a This patch turns the Python API mismatch notice into a standard Marc-André Lemburg <mal@egenix.com> parents: diff changeset	72
21145 87fa6e9bd77e mysnprintf.c: Massive rewrite of PyOS_snprintf and PyOS_vsnprintf, to Tim Peters <tim.peters@gmail.com> parents: 21126 diff changeset	73 #ifdef HAVE_SNPRINTF
61466 cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52976 diff changeset	74 len = vsnprintf(str, size, format, va);
21145 87fa6e9bd77e mysnprintf.c: Massive rewrite of PyOS_snprintf and PyOS_vsnprintf, to Tim Peters <tim.peters@gmail.com> parents: 21126 diff changeset	75 #else
61466 cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52976 diff changeset	76 /* Emulate it. */
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52976 diff changeset	77 buffer = PyMem_MALLOC(size + _PyOS_vsnprintf_EXTRA_SPACE);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52976 diff changeset	78 if (buffer == NULL) {
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52976 diff changeset	79 len = -666;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52976 diff changeset	80 goto Done;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52976 diff changeset	81 }
18908 79c676f9b34a This patch turns the Python API mismatch notice into a standard Marc-André Lemburg <mal@egenix.com> parents: diff changeset	82
61466 cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52976 diff changeset	83 len = vsprintf(buffer, format, va);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52976 diff changeset	84 if (len < 0)
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52976 diff changeset	85 /* ignore the error */;
21145 87fa6e9bd77e mysnprintf.c: Massive rewrite of PyOS_snprintf and PyOS_vsnprintf, to Tim Peters <tim.peters@gmail.com> parents: 21126 diff changeset	86
61466 cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52976 diff changeset	87 else if ((size_t)len >= size + _PyOS_vsnprintf_EXTRA_SPACE)
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52976 diff changeset	88 Py_FatalError("Buffer overflow in PyOS_snprintf/PyOS_vsnprintf");
18908 79c676f9b34a This patch turns the Python API mismatch notice into a standard Marc-André Lemburg <mal@egenix.com> parents: diff changeset	89
61466 cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52976 diff changeset	90 else {
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52976 diff changeset	91 const size_t to_copy = (size_t)len < size ?
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52976 diff changeset	92 (size_t)len : size - 1;
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52976 diff changeset	93 assert(to_copy < size);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52976 diff changeset	94 memcpy(str, buffer, to_copy);
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52976 diff changeset	95 str[to_copy] = '\0';
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52976 diff changeset	96 }
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52976 diff changeset	97 PyMem_FREE(buffer);
47559 2925b568aacc Merged revisions 63562,63570,63728,63734,63784,63788,63802,63817,63827,63839,63887,63975,63998 via svnmerge from Georg Brandl <georg@python.org> parents: 21518 diff changeset	98 #endif
21145 87fa6e9bd77e mysnprintf.c: Massive rewrite of PyOS_snprintf and PyOS_vsnprintf, to Tim Peters <tim.peters@gmail.com> parents: 21126 diff changeset	99 Done:
61466 cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52976 diff changeset	100 if (size > 0)
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52976 diff changeset	101 str[size-1] = '\0';
cab14be0ada1 Recorded merge of revisions 81029 via svnmerge from Antoine Pitrou <solipsis@pitrou.net> parents: 52976 diff changeset	102 return len;
47559 2925b568aacc Merged revisions 63562,63570,63728,63734,63784,63788,63802,63817,63827,63839,63887,63975,63998 via svnmerge from Georg Brandl <georg@python.org> parents: 21518 diff changeset	103 #undef _PyOS_vsnprintf_EXTRA_SPACE
18908 79c676f9b34a This patch turns the Python API mismatch notice into a standard Marc-André Lemburg <mal@egenix.com> parents: diff changeset	104 }

Mercurial > cpython-lru_cache-weakref

annotate Python/mysnprintf.c @ 87690:66c1c9f32567 default tip