cpython-lru_cache-weakref: Include/stringobject.h history

Dummy-merge 2.6 branch into 2.7 branch.

Sat, 05 Mar 2011 15:15:07 +0100

Dummy-merge 2.6 branch into 2.7 branch.

Backport of some of the work in r71665 to trunk. This reworks much of

Wed, 22 Apr 2009 13:29:05 +0000

Backport of some of the work in r71665 to trunk. This reworks much of
int, long, and float __format__(), and it keeps their implementation
in sync with py3k.

Also added PyOS_double_to_string. This is the "fallback" version
that's also available in trunk, and should be kept in sync with that
code. I'll add an issue to document PyOS_double_to_string in the C
API.

There are many internal cleanups. Externally visible changes include:

- Implement PEP 378, Format Specifier for Thousands Separator, for
floats, ints, and longs.

- Issue #5515: 'n' formatting for ints, longs, and floats handles
leading zero formatting poorly.

- Issue #5772: For float.__format__, don't add a trailing ".0" if
we're using no type code and we have an exponent.

Recorded merge of revisions 81029 via svnmerge from

Sun, 09 May 2010 15:15:40 +0000

Recorded merge of revisions 81029 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines

Untabify C files. Will watch buildbots.
........

Modified interface to _Py_[String|Unicode]InsertThousandsGrouping, in anticipation of fixing issue 3140.

Tue, 24 Jun 2008 00:42:10 +0000

Modified interface to _Py_[String|Unicode]InsertThousandsGrouping, in anticipation of fixing issue 3140.

swap stringobject.h and bytesobject.h contents to make sense. PyString in

Wed, 11 Jun 2008 03:40:10 +0000

swap stringobject.h and bytesobject.h contents to make sense. PyString in
stringobject and PyBytes defines in bytesobject.

This reverts r63675 based on the discussion in this thread:

Mon, 09 Jun 2008 04:58:54 +0000

This reverts r63675 based on the discussion in this thread:

http://mail.python.org/pipermail/python-dev/2008-June/079988.html

Python 2.6 should stick with PyString_* in its codebase. The PyBytes_* names
in the spirit of 3.0 are available via a #define only. See the email thread.

Renamed PyString to PyBytes

Mon, 26 May 2008 12:51:38 +0000

Renamed PyString to PyBytes

First step of the C API rename:

Mon, 26 May 2008 12:29:14 +0000

First step of the C API rename:
renamed Include/bytesobject.h to Include/bytearrayobject.h
renamed Include/stringobject.h to Include/bytesobject.h
added Include/stringobject.h with aliases

Addresses issue 2802: 'n' formatting for integers.

Sun, 11 May 2008 19:52:48 +0000

Addresses issue 2802: 'n' formatting for integers.

Adds 'n' as a format specifier for integers, to mirror the same
specifier which is already available for floats. 'n' is the same as
'd', but inserts the current locale-specific thousands grouping.

I added this as a stringlib function, but it's only used by str type,
not unicode. This is because of an implementation detail in
unicode.format(), which does its own str->unicode conversion. But the
unicode version will be needed in 3.0, and it may be needed by other
code eventually in 2.6 (maybe decimal?), so I left it as a stringlib
implementation. As long as the unicode version isn't instantiated,
there's no overhead for this.

#1629: Renamed Py_Size, Py_Type and Py_Refcnt to Py_SIZE, Py_TYPE and Py_REFCNT. Macros for b/w compatibility are available.

Wed, 19 Dec 2007 02:37:44 +0000

#1629: Renamed Py_Size, Py_Type and Py_Refcnt to Py_SIZE, Py_TYPE and Py_REFCNT. Macros for b/w compatibility are available.

PEP 3123: Provide forward compatibility with Python 3.0, while keeping

Sat, 21 Jul 2007 06:55:02 +0000

PEP 3123: Provide forward compatibility with Python 3.0, while keeping
backwards compatibility. Add Py_Refcnt, Py_Type, Py_Size, and
PyVarObject_HEAD_INIT.

Variation of patch # 1624059 to speed up checking if an object is a subclass

Sun, 25 Feb 2007 19:44:48 +0000

Variation of patch # 1624059 to speed up checking if an object is a subclass
of some of the common builtin types.

Use a bit in tp_flags for each common builtin type. Check the bit
to determine if any instance is a subclass of these common types.
The check avoids a function call and O(n) search of the base classes.
The check is done in the various Py*_Check macros rather than calling
PyType_IsSubtype().

All the bits are set in tp_flags when the type is declared
in the Objects/*object.c files because PyType_Ready() is not called
for all the types. Should PyType_Ready() be called for all types?
If so and the change is made, the changes to the Objects/*object.c files
can be reverted (remove setting the tp_flags). Objects/typeobject.c
would also have to be modified to add conditions
for Py*_CheckExact() in addition to each the PyType_IsSubtype check.

Renamed PyString to PyBytes

Mon, 26 May 2008 13:28:38 +0000

Renamed PyString to PyBytes

Merged revisions 63078 via svnmerge from

Sun, 11 May 2008 21:00:57 +0000

Merged revisions 63078 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

When forward porting this, I added _PyUnicode_InsertThousandsGrouping.

........
r63078 | eric.smith | 2008-05-11 15:52:48 -0400 (Sun, 11 May 2008) | 14 lines

Addresses issue 2802: 'n' formatting for integers.

Adds 'n' as a format specifier for integers, to mirror the same
specifier which is already available for floats. 'n' is the same as
'd', but inserts the current locale-specific thousands grouping.

I added this as a stringlib function, but it's only used by str type,
not unicode. This is because of an implementation detail in
unicode.format(), which does its own str->unicode conversion. But the
unicode version will be needed in 3.0, and it may be needed by other
code eventually in 2.6 (maybe decimal?), so I left it as a stringlib
implementation. As long as the unicode version isn't instantiated,
there's no overhead for this.
........

#1629: Renamed Py_Size, Py_Type and Py_Refcnt to Py_SIZE, Py_TYPE and Py_REFCNT.

Wed, 19 Dec 2007 02:45:37 +0000

#1629: Renamed Py_Size, Py_Type and Py_Refcnt to Py_SIZE, Py_TYPE and Py_REFCNT.

Fixed issue #1564: The set implementation should special-case PyUnicode instead of PyString

Mon, 10 Dec 2007 15:50:56 +0000

Fixed issue #1564: The set implementation should special-case PyUnicode instead of PyString
I moved the unicode_eq to stringlib/eq.h to keep the function static and possible inline for setobject.c and dictobject.h. I also removed the unused _PyString_Eq method. If it's required in the future it can be added to eq.h

Added all PyTypeObjects to the appropriate header files.

Thu, 29 Nov 2007 22:35:39 +0000

Added all PyTypeObjects to the appropriate header files.
Before the patch a lot of internal types weren't available in the header files. The patch exposes the new iterators, views and some other types to all C modules. I've also renamed some of the types and tp_names.

Merging the py3k-pep3137 branch back into the py3k branch.

Tue, 06 Nov 2007 21:34:58 +0000

Merging the py3k-pep3137 branch back into the py3k branch.
No detailed change log; just check out the change log for the py3k-pep3137
branch. The most obvious changes:

- str8 renamed to bytes (PyString at the C level);
- bytes renamed to buffer (PyBytes at the C level);
- PyString and PyUnicode are no longer compatible.

I.e. we now have an immutable bytes type and a mutable bytes type.

The behavior of PyString was modified quite a bit, to make it more
bytes-like. Some changes are still on the to-do list.

Kill PyString_Encode(), which isn't used anywhere.

Fri, 26 Oct 2007 19:34:40 +0000

Kill PyString_Encode(), which isn't used anywhere.
(We should also kill the other PyString_ encode/decode APIs, but they
still have some users.)

Patch# 1258 by Christian Heimes: kill basestring.

Tue, 16 Oct 2007 18:12:55 +0000

Patch# 1258 by Christian Heimes: kill basestring.
I like this because it makes the code shorter! :-)

Fix merge breakage.

Sat, 21 Jul 2007 18:47:48 +0000

Fix merge breakage.

Merged revisions 56467-56482 via svnmerge from

Sat, 21 Jul 2007 17:22:18 +0000

Merged revisions 56467-56482 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/p3yk

................
r56477 | martin.v.loewis | 2007-07-21 09:04:38 +0200 (Sa, 21 Jul 2007) | 11 lines

Merged revisions 56466-56476 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
r56476 | martin.v.loewis | 2007-07-21 08:55:02 +0200 (Sa, 21 Jul 2007) | 4 lines

PEP 3123: Provide forward compatibility with Python 3.0, while keeping
backwards compatibility. Add Py_Refcnt, Py_Type, Py_Size, and
PyVarObject_HEAD_INIT.
........
................
r56478 | martin.v.loewis | 2007-07-21 09:47:23 +0200 (Sa, 21 Jul 2007) | 2 lines

PEP 3123: Use proper C inheritance for PyObject.
................
r56479 | martin.v.loewis | 2007-07-21 10:06:55 +0200 (Sa, 21 Jul 2007) | 3 lines

Add longintrepr.h to Python.h, so that the compiler can
see that PyFalse is really some kind of PyObject*.
................
r56480 | martin.v.loewis | 2007-07-21 10:47:18 +0200 (Sa, 21 Jul 2007) | 2 lines

Qualify SHIFT, MASK, BASE.
................
r56482 | martin.v.loewis | 2007-07-21 19:10:57 +0200 (Sa, 21 Jul 2007) | 2 lines

Correctly refer to _ob_next.
................

Make identifiers str (not str8) objects throughout.

Sun, 10 Jun 2007 09:51:05 +0000

Make identifiers str (not str8) objects throughout.
This affects the parser, various object implementations,
and all places that put identifiers into C string literals.

In testing, a number of crashes occurred as code would
fail when the recursion limit was reached (such as the
Unicode interning dictionary having key/value pairs where
key is not value). To solve these, I added an overflowed
flag, which allows for 50 more recursions after the
limit was reached and the exception was raised, and
a recursion_critical flag, which indicates that recursion
absolutely must be allowed, i.e. that a certain call
must not cause a stack overflow exception.

There are still some places where both str and str8 are
accepted as identifiers; these should eventually be
removed.

Add interning of unicode strings by copying the functionality from

Fri, 25 May 2007 13:52:07 +0000

Add interning of unicode strings by copying the functionality from
stringobject.c.

Intern "True" and "False" in bool_repr() again as it was in the
8bit string era.

Merged revisions 53875-53911 via svnmerge from

Sun, 25 Feb 2007 20:39:11 +0000

Merged revisions 53875-53911 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
r53899 | neal.norwitz | 2007-02-25 16:52:27 +0100 (Sun, 25 Feb 2007) | 1 line

Add more details when releasing interned strings
........
r53900 | neal.norwitz | 2007-02-25 16:53:36 +0100 (Sun, 25 Feb 2007) | 1 line

Whitespace only changes
........
r53901 | jeremy.hylton | 2007-02-25 16:57:45 +0100 (Sun, 25 Feb 2007) | 8 lines

Fix crash in exec when unicode filename can't be decoded.

I can't think of an easy way to test this behavior. It only occurs
when the file system default encoding and the interpreter default
encoding are different, such that you can open the file but not decode
its name.
........
r53902 | jeremy.hylton | 2007-02-25 17:01:58 +0100 (Sun, 25 Feb 2007) | 2 lines

Put declarations before code.
........
r53910 | fred.drake | 2007-02-25 18:56:27 +0100 (Sun, 25 Feb 2007) | 3 lines

- SF patch #1657613: add documentation for the Element interface
- clean up bogus use of the {datadescni} environment everywhere
........
r53911 | neal.norwitz | 2007-02-25 20:44:48 +0100 (Sun, 25 Feb 2007) | 17 lines

Variation of patch # 1624059 to speed up checking if an object is a subclass
of some of the common builtin types.

Use a bit in tp_flags for each common builtin type. Check the bit
to determine if any instance is a subclass of these common types.
The check avoids a function call and O(n) search of the base classes.
The check is done in the various Py*_Check macros rather than calling
PyType_IsSubtype().

All the bits are set in tp_flags when the type is declared
in the Objects/*object.c files because PyType_Ready() is not called
for all the types. Should PyType_Ready() be called for all types?
If so and the change is made, the changes to the Objects/*object.c files
can be reverted (remove setting the tp_flags). Objects/typeobject.c
would also have to be modified to add conditions
for Py*_CheckExact() in addition to each the PyType_IsSubtype check.
........

Patch #1601678: move intern() to sys.intern().

Tue, 19 Dec 2006 20:50:34 +0000

Patch #1601678: move intern() to sys.intern().

Merge ssize_t branch.

Wed, 15 Feb 2006 17:27:45 +0000

Merge ssize_t branch.

Wrote down the invariants of some common objects whose structure is

Thu, 28 Oct 2004 16:32:00 +0000

Wrote down the invariants of some common objects whose structure is
exposed in header files. Fixed a few comments in these headers.

As we might have expected, writing down invariants systematically exposed a
(minor) bug. In this case, function objects have a writeable func_code
attribute, which could be set to code objects with the wrong number of
free variables. Calling the resulting function segfaulted the interpreter.
Added a corresponding test.

Use appropriate macros not the deprecated DL_IMPORT/DL_EXPORT macros

Tue, 01 Jul 2003 20:15:21 +0000

Use appropriate macros not the deprecated DL_IMPORT/DL_EXPORT macros

Use Py_GCC_ATTRIBUTE instead of attribute. Compilers other than GCC

Sun, 15 Sep 2002 14:09:54 +0000

Use Py_GCC_ATTRIBUTE instead of __attribute__. Compilers other than GCC
might use __attribute__ in other ways (e.g. CodeWarrior).

SF patch 576101, by Oren Tirosh: alternative implementation of

Mon, 19 Aug 2002 21:43:18 +0000

SF patch 576101, by Oren Tirosh: alternative implementation of
interning. I modified Oren's patch significantly, but the basic idea
and most of the implementation is unchanged. Interned strings created
with PyString_InternInPlace() are now mortal, and you must keep a
reference to the resulting string around; use the new function
PyString_InternImmortal() to create immortal interned strings.

Patch #505705: Remove eval in pickle and cPickle.

Wed, 14 Aug 2002 07:46:28 +0000

Patch #505705: Remove eval in pickle and cPickle.

Excise DL_EXPORT from Include.

Mon, 12 Aug 2002 07:21:58 +0000

Excise DL_EXPORT from Include.

Thanks to Skip Montanaro and Kalle Svensson for the patches.

- A new type object, 'string', is added. This is a common base type

Fri, 24 May 2002 19:01:59 +0000

- A new type object, 'string', is added. This is a common base type
for 'str' and 'unicode', and can be used instead of
types.StringTypes, e.g. to test whether something is "a string":
isinstance(x, string) is True for Unicode and 8-bit strings. This
is an abstract base class and cannot be instantiated directly.

Remove the CACHE_HASH and INTERN_STRINGS preprocessor symbols.

Fri, 29 Mar 2002 03:29:08 +0000

Remove the CACHE_HASH and INTERN_STRINGS preprocessor symbols.

Add function attributes that allow GCC to check the arguments of printf-like

Tue, 23 Oct 2001 02:21:22 +0000

Add function attributes that allow GCC to check the arguments of printf-like
functions.

More on SF bug [#460020] bug or feature: unicode() and subclasses.

Tue, 11 Sep 2001 01:41:59 +0000

More on SF bug [#460020] bug or feature: unicode() and subclasses.
Repaired str(i) to return a genuine string when i is an instance of a str
subclass. New PyString_CheckExact() macro.

Make the Py_Check() macro use PyObject_TypeCheck().

Thu, 30 Aug 2001 03:08:07 +0000

Make the Py_Check() macro use PyObject_TypeCheck().

PyString_FromFormat() and PyString_FromFormatV(): Largely ripped from

Fri, 24 Aug 2001 18:32:06 +0000

PyString_FromFormat() and PyString_FromFormatV(): Largely ripped from
PyErr_Format() these new C API methods can be used instead of
sprintf()'s into hardcoded char* buffers. This allows us to fix
many situation where long package, module, or class names get
truncated in reprs.

PyString_FromFormat() is the varargs variety.
PyString_FromFormatV() is the va_list variety

Original PyErr_Format() code was modified to allow %p and %ld
expansions.

Many reprs were converted to this, checkins coming soo. Not
changed: complex_repr(), float_repr(), float_print(), float_str(),
int_repr(). There may be other candidates not yet converted.

Closes patch #454743.

SF bug 433228: repr(list) woes when len(list) big.

Sat, 16 Jun 2001 05:11:17 +0000

SF bug 433228: repr(list) woes when len(list) big.
Gave Python linear-time repr() implementations for dicts, lists, strings.
This means, e.g., that repr(range(50000)) is no longer 50x slower than
pprint.pprint() in 2.2 .

I don't consider this a bugfix candidate, as it's a performance boost.

Added _PyString_Join() to the internal string API. If we want that in the
public API, fine, but then it requires runtime error checks instead of
asserts.

Patch #424335: Implement string_richcompare, remove string_compare.

Thu, 24 May 2001 16:56:35 +0000

Patch #424335: Implement string_richcompare, remove string_compare.
Use new _PyString_Eq in lookdict_string.

This patch changes the way the string .encode() method works slightly

Tue, 15 May 2001 12:00:02 +0000

This patch changes the way the string .encode() method works slightly
and introduces a new method .decode().

The major change is that strg.encode() will no longer try to convert
Unicode returns from the codec into a string, but instead pass along
the Unicode object as-is. The same is now true for all other codec
return types. The underlying C APIs were changed accordingly.

Note that even though this does have the potential of breaking
existing code, the chances are low since conversion from Unicode
previously took place using the default encoding which is normally
set to ASCII rendering this auto-conversion mechanism useless for
most Unicode encodings.

The good news is that you can now use .encode() and .decode() with
much greater ease and that the door was opened for better accessibility
of the builtin codecs.

As demonstration of the new feature, the patch includes a few new
codecs which allow string to string encoding and decoding (rot13,
hex, zip, uu, base64).

Written by Marc-Andre Lemburg. Copyright assigned to the PSF.

_Py_ReleaseInternedStrings(): Private API function to decref and

Fri, 23 Feb 2001 16:40:48 +0000

_Py_ReleaseInternedStrings(): Private API function to decref and
release the interned string dictionary. This is useful for memory
use debugging because it eliminates a huge source of noise from the
reports. Only defined when INTERN_STRINGS is defined.

Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.

Thu, 21 Sep 2000 05:43:11 +0000

Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Note a curious extension to the std C rules: x, X and o formatting can never produce
a sign character in C, so the '+' and ' ' flags are meaningless for them. But
unbounded ints *can* produce a sign character under these conversions (no fixed-
width bitstring is wide enough to hold all negative values in 2's-comp form). So
these flags become meaningful in Python when formatting a Python long which is too
big to fit in a C long. This required shuffling around existing code, which hacked
x and X conversions to death when both the '#' and '0' flags were specified: the
hacks weren't strong enough to deal with the simultaneous possibility of the ' ' or
'+' flags too, since signs were always meaningless before for x and X conversions.
Isomorphic shuffling was required in unicodeobject.c.
Also added dozens of non-trivial new unbounded-int test cases to test_format.py.

This patch adds a new Python C API called PyString_AsStringAndSize()

Tue, 19 Sep 2000 21:04:18 +0000

This patch adds a new Python C API called PyString_AsStringAndSize()
which implements the automatic conversion from Unicode to a string
object using the default encoding.

The new API is then put to use to have eval() and exec accept
Unicode objects as code parameter. This closes bugs #110924
and #113890.

As side-effect, the traditional C APIs PyString_Size() and
PyString_AsString() will also accept Unicode objects as
parameters.

REMOVED all CWI, CNRI and BeOpen copyright markings.

Fri, 01 Sep 2000 23:29:29 +0000

REMOVED all CWI, CNRI and BeOpen copyright markings.
This should match the situation in the 1.6b1 tree.

ANSI-fication and Py_PROTO extermination.

Sun, 09 Jul 2000 00:55:06 +0000

ANSI-fication and Py_PROTO extermination.

Added prototypes for the new codec APIs for strings. These APIs

Thu, 06 Jul 2000 11:25:40 +0000

Added prototypes for the new codec APIs for strings. These APIs
match the ones in the Unicode implementation, but were extended
to be able to reuse the existing Unicode codecs for string
purposes too.

Conversion from string to Unicode and back are done using the
default encoding.

Change copyright notice - 2nd try.

Fri, 30 Jun 2000 23:58:06 +0000

Change copyright notice - 2nd try.

Change copyright notice.

Fri, 30 Jun 2000 23:50:40 +0000

Change copyright notice.

Add DL_IMPORT(returntype) for all officially exported functions.

Fri, 04 Dec 1998 18:48:25 +0000

Add DL_IMPORT(returntype) for all officially exported functions.

Two speedup hacks. Caching the hash saves recalculation of a string's

Sat, 18 Jan 1997 07:53:23 +0000

Two speedup hacks. Caching the hash saves recalculation of a string's
hash value. Interning strings (which requires hash caching) tries to
ensure that only one string object with a given value exists, so
equality tests are one pointer comparison. Together, these can speed
the interpreter up by as much as 20%. Each costs the size of a long
or pointer per string object. In addition, interned strings live
until the end of times. If you are concerned about memory footprint,
simply comment the #define out here (and rebuild everything!).

added PyString_GET_SIZE macro

Mon, 06 Jan 1997 22:42:50 +0000

added PyString_GET_SIZE macro

for both PyString_GET_SIZE and PyString_AS_STRING, cast first argument
to a PyStringObject*

Add const to error and newstring functions

Tue, 10 Dec 1996 15:33:34 +0000

Add const to error and newstring functions

New permission notice, includes CNRI.

Fri, 25 Oct 1996 14:44:06 +0000

New permission notice, includes CNRI.

Turn on CACHE_HASH, for 2% speedier dict lookups

Tue, 30 Jul 1996 16:42:03 +0000

Turn on CACHE_HASH, for 2% speedier dict lookups

make the type a parameter of the DL_IMPORT macro, for Borland C

Mon, 27 Feb 1995 10:17:52 +0000

make the type a parameter of the DL_IMPORT macro, for Borland C

new names for lots of new functions

Tue, 17 Jan 1995 16:01:01 +0000

new names for lots of new functions

The great renaming, phase two: all header files have been updated to

Thu, 12 Jan 1995 11:45:45 +0000

The great renaming, phase two: all header files have been updated to
use the new names exclusively, and the linker will see the new names.
Files that import "Python.h" also only see the new names. Files that
import "allobjects.h" will continue to be able to use the old names,
due to the inclusion (in allobjects.h) of "rename2.h".

Wed, 04 Jan 1995 19:06:22 +0000

Added 1995 copyright.
object.h: made sizes and refcnts signed ints.
stringobject.h: make getstrsize() signed int.
methodobject.h: add METH_VARARGS and METH_FREENAME flag bit definitions.