https://hg.dopieralski.pl/cpython-lru_cache-weakref/ en-us Modules/_pickle.c revision history https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/7d6c27fa7f32/Modules/_pickle.c Alexandre Vassalotti <alexandre@peadrop.com> Sat, 30 Nov 2013 17:58:53 -0800 https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/ff56f48b3277/Modules/_pickle.c Alexandre Vassalotti <alexandre@peadrop.com> Sat, 30 Nov 2013 16:21:20 -0800 https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/f20d966dc499/Modules/_pickle.c Alexandre Vassalotti <alexandre@peadrop.com> Sat, 30 Nov 2013 13:55:39 -0800 https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/d565310e7ae3/Modules/_pickle.c Serhiy Storchaka <storchaka@gmail.com> Sat, 30 Nov 2013 23:15:38 +0200 https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/d829111a27e5/Modules/_pickle.c Alexandre Vassalotti <alexandre@peadrop.com> Thu, 28 Nov 2013 17:09:16 -0800 https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/b44dbf07732e/Modules/_pickle.c
PyTuple_New(0) always returns the same empty tuple from its free list anyway,
so we are not saving much here. Plus, the code where this was used is on
uncommon run paths.]]> Alexandre Vassalotti <alexandre@peadrop.com> Thu, 28 Nov 2013 15:17:29 -0800 https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/dd51b72cfb52/Modules/_pickle.c
I have noticed a race-condition occurring on one of the buildbots because of
this optimization. The function called may release the GIL which means
multiple threads may end up accessing the shared tuple. I could fix it up by
storing the tuple to the Pickler and Unipickler object again, but honestly it
really not worth the trouble.

I ran many benchmarks and the only time the optimization helps is when using a
fin-memory file, like io.BytesIO on which reads are super cheap, combined with
pickle protocol less than 4. Even in this contrived case, the speedup is a
about 5%. For everything else, this optimization does not provide any
noticable improvements.]]> Alexandre Vassalotti <alexandre@peadrop.com> Thu, 28 Nov 2013 14:56:09 -0800 https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/64c6d52793be/Modules/_pickle.c
This implements PEP 3121 module finalization as well. This change does not
cause any significant impact on performance.]]> Alexandre Vassalotti <alexandre@peadrop.com> Wed, 27 Nov 2013 19:36:52 -0800 https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/23459df0753e/Modules/_pickle.c
I fixed the bug that was in my previous attempt of this cleanup. I ran
the full test suite to verify I didn't introduce any obvious bugs.]]> Alexandre Vassalotti <alexandre@peadrop.com> Wed, 27 Nov 2013 02:26:54 -0800 https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/2a1de922651a/Modules/_pickle.c Alexandre Vassalotti <alexandre@peadrop.com> Mon, 25 Nov 2013 13:25:12 -0800 https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/e39db21df580/Modules/_pickle.c Alexandre Vassalotti <alexandre@peadrop.com> Mon, 25 Nov 2013 13:03:32 -0800 https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/f8ac01a762c1/Modules/_pickle.c Alexandre Vassalotti <alexandre@peadrop.com> Mon, 25 Nov 2013 11:35:46 -0800 https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/14f2776686b3/Modules/_pickle.c
Also, replace unnessary uses of the #if preprocessor directive.]]> Alexandre Vassalotti <alexandre@peadrop.com> Sun, 24 Nov 2013 22:41:13 -0800 https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/f27d6721f446/Modules/_pickle.c Alexandre Vassalotti <alexandre@peadrop.com> Sun, 24 Nov 2013 21:40:18 -0800 https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/ead3f5a907bd/Modules/_pickle.c
This doesn't make any functional changes to the exisiting implementation. The
conversion did help however uncover documentation bugs. The best thing about
this conversion is less C code to maintain by hand.]]> Alexandre Vassalotti <alexandre@peadrop.com> Sun, 24 Nov 2013 12:25:48 -0800 https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/6bd1f0a27e8e/Modules/_pickle.c Alexandre Vassalotti <alexandre@peadrop.com> Sun, 24 Nov 2013 03:07:35 -0800 https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/04ccb2e0307e/Modules/_pickle.c Alexandre Vassalotti <alexandre@peadrop.com> Sun, 24 Nov 2013 02:53:45 -0800 https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/24c84d00cf1f/Modules/_pickle.c Alexandre Vassalotti <alexandre@peadrop.com> Sun, 24 Nov 2013 02:41:05 -0800 https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/de9bda43d552/Modules/_pickle.c
This will allow us to control in the future whether to use framing or not.
For example, we may want to turn it off for tiny pickle where it doesn't
help.

The change also improves performance slightly:

### fastpickle ###
Min: 0.608517 -> 0.557358: 1.09x faster
Avg: 0.798892 -> 0.694738: 1.15x faster
Significant (t=3.45)
Stddev: 0.17145 -> 0.12704: 1.3496x smaller
Timeline: http://goo.gl/3xQE1J

### pickle_dict ###
Min: 0.669920 -> 0.615271: 1.09x faster
Avg: 0.733633 -> 0.645058: 1.14x faster
Significant (t=5.05)
Stddev: 0.12041 -> 0.02961: 4.0662x smaller
Timeline: http://goo.gl/LpLSXI

### pickle_list ###
Min: 0.397583 -> 0.368112: 1.08x faster
Avg: 0.412784 -> 0.397223: 1.04x faster
Significant (t=2.78)
Stddev: 0.01518 -> 0.03653: 2.4068x larger
Timeline: http://goo.gl/v39E59

### unpickle_list ###
Min: 0.692935 -> 0.594870: 1.16x faster
Avg: 0.730012 -> 0.628395: 1.16x faster
Significant (t=17.76)
Stddev: 0.02720 -> 0.02995: 1.1012x larger
Timeline: http://goo.gl/2P9AEt

The following not significant results are hidden, use -v to show them:
fastunpickle.]]> Alexandre Vassalotti <alexandre@peadrop.com> Sat, 23 Nov 2013 20:30:03 -0800 https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/c3fd79b17983/Modules/_pickle.c Antoine Pitrou <solipsis@pitrou.net> Sat, 23 Nov 2013 21:34:04 +0100 https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/6eca53cf005d/Modules/_pickle.c and warns about potential uninitialized use.
Silence that by initializing it to null.]]> Gregory P. Smith <greg@krypto.org> Sat, 23 Nov 2013 20:21:28 +0000 https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/a02adfb3260a/Modules/_pickle.c CID 1131946: Unchecked return value (CHECKED_RETURN)]]> Christian Heimes <christian@cheimes.de> Sat, 23 Nov 2013 21:13:39 +0100 https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/3c222d1f81e9/Modules/_pickle.c Antoine Pitrou <solipsis@pitrou.net> Sat, 23 Nov 2013 21:06:21 +0100 https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/458340ed0606/Modules/_pickle.c Antoine Pitrou <solipsis@pitrou.net> Sat, 23 Nov 2013 21:05:08 +0100 https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/c54becd69805/Modules/_pickle.c Christian Heimes <christian@cheimes.de> Sat, 23 Nov 2013 21:05:31 +0100 https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/d719975f4d25/Modules/_pickle.c CID 1131949: Dereference null return value (NULL_RETURNS)]]> Christian Heimes <christian@cheimes.de> Sat, 23 Nov 2013 21:01:40 +0100 https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/992ef855b3ed/Modules/_pickle.c Most of the work is by Alexandre.]]> Antoine Pitrou <solipsis@pitrou.net> Sat, 23 Nov 2013 18:59:12 +0100 https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/7d0323556c53/Modules/_pickle.c error handling

The following code didn't handle correctly the failure of
PyUnicode_InternFromString("__name__").

if (newobj_str == NULL) {
newobj_str = PyUnicode_InternFromString("__newobj__");
name_str = PyUnicode_InternFromString("__name__");
if (newobj_str == NULL || name_str == NULL)
return -1;
}]]> Victor Stinner <victor.stinner@gmail.com> Thu, 14 Nov 2013 01:26:17 +0100 https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/6348764bacdd/Modules/_pickle.c "modules" once]]> Victor Stinner <victor.stinner@gmail.com> Wed, 06 Nov 2013 22:40:41 +0100 https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/d191d22a9d23/Modules/_pickle.c exception set]]> Victor Stinner <victor.stinner@gmail.com> Thu, 31 Oct 2013 13:39:23 +0100 https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/78b35cf9c024/Modules/_pickle.c Victor Stinner <victor.stinner@gmail.com> Thu, 31 Oct 2013 13:38:42 +0100 https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/0872257752c2/Modules/_pickle.c error messages and comments.]]> Serhiy Storchaka <storchaka@gmail.com> Tue, 27 Aug 2013 20:17:03 +0300 https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/b33f81974516/Modules/_pickle.c Christian Heimes <christian@cheimes.de> Fri, 26 Jul 2013 22:45:47 +0200 https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/6bd01a59762a/Modules/_pickle.c Victor Stinner <victor.stinner@gmail.com> Tue, 16 Jul 2013 22:26:05 +0200 https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/bb5da6e795a1/Modules/_pickle.c Victor Stinner <victor.stinner@gmail.com> Fri, 12 Jul 2013 00:53:57 +0200 https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/f85fcbbbe8de/Modules/_pickle.c allocating a bigger table failed

PyMemoTable destructor does crash if mt_table is NULL.]]> Victor Stinner <victor.stinner@gmail.com> Fri, 12 Jul 2013 00:53:26 +0200 https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/1eac89af9acf/Modules/_pickle.c the newly created pickler, and not PyObject_GC_Del().]]> Victor Stinner <victor.stinner@gmail.com> Fri, 12 Jul 2013 00:08:59 +0200 https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/1a1869baec4c/Modules/_pickle.c attributes before handling errors

_Pickler_New() now calls PyObject_GC_Del() instead of Py_DECREF() on error,
because the pickle object is created using PyObject_GC_New().

Fix a crash in the destructor when an attribute is not initiallized.]]> Victor Stinner <victor.stinner@gmail.com> Thu, 11 Jul 2013 22:56:25 +0200 https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/65f2c92ed079/Modules/_pickle.c
Replace strdup() with _PyMem_RawStrdup() or _PyMem_Strdup(), depending if the
GIL is held or not.]]> Victor Stinner <victor.stinner@gmail.com> Sun, 07 Jul 2013 23:30:24 +0200 https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/f17647ce8f8a/Modules/_pickle.c Christian Heimes <christian@cheimes.de> Mon, 01 Jul 2013 15:23:48 +0200 https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/f3372692ca20/Modules/_pickle.c segfault inside the _pickle C extension.]]> Christian Heimes <christian@cheimes.de> Mon, 01 Jul 2013 15:18:49 +0200 https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/ac7bc6700ac3/Modules/_pickle.c CID 983309 (#1 of 1): Resource leak (RESOURCE_LEAK)
leaked_storage: Variable unicode_str going out of scope leaks the storage it points to.]]> Christian Heimes <christian@cheimes.de> Sat, 29 Jun 2013 21:37:34 +0200 https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/251c5b4bb05d/Modules/_pickle.c Alexandre Vassalotti <alexandre@peadrop.com> Sat, 20 Apr 2013 21:28:49 -0700 https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/4a6b8f86b081/Modules/_pickle.c
The C version tried to remove trailing whitespace between the last quote and
the newline character. I am not sure why it had this because pickle never
generated such pickles---for this to happen repr(some_string) would need to
return trailing whitespace. It was maybe there to make it easier for people
to write pickles in text editors. Anyhow, the Python version doesn't do this
so there is no point keeping this around anymore.

Also, I've changed the exception raised when a bad pickle is encountered.
Again this unlikely to make much difference to anyone though it does make
testing slightly nicer for us.]]> Alexandre Vassalotti <alexandre@peadrop.com> Mon, 15 Apr 2013 23:14:55 -0700 https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/5a16d2992112/Modules/_pickle.c Antoine Pitrou <solipsis@pitrou.net> Mon, 15 Apr 2013 21:55:14 +0200 https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/686be49a1c56/Modules/_pickle.c Alexandre Vassalotti <alexandre@peadrop.com> Sun, 14 Apr 2013 02:25:10 -0700 https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/7bf4aca65ba2/Modules/_pickle.c Alexandre Vassalotti <alexandre@peadrop.com> Sun, 14 Apr 2013 02:11:16 -0700 https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/0437c19cdad0/Modules/_pickle.c Antoine Pitrou <solipsis@pitrou.net> Sun, 07 Apr 2013 23:46:52 +0200 https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/09a84091ae96/Modules/_pickle.c Antoine Pitrou <solipsis@pitrou.net> Sun, 07 Apr 2013 17:38:11 +0200 https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/9fcba15d7685/Modules/_pickle.c Alexandre Vassalotti <alexandre@peadrop.com> Sat, 30 Nov 2013 17:55:48 -0800 https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/16eba94d3cfe/Modules/_pickle.c Alexandre Vassalotti <alexandre@peadrop.com> Sat, 30 Nov 2013 16:06:39 -0800 https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/b92f9eaedb76/Modules/_pickle.c Alexandre Vassalotti <alexandre@peadrop.com> Sat, 30 Nov 2013 13:52:35 -0800 https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/4d62a62ba44d/Modules/_pickle.c error messages and comments.]]> Serhiy Storchaka <storchaka@gmail.com> Tue, 27 Aug 2013 19:40:23 +0300 https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/dbdd07657e23/Modules/_pickle.c Christian Heimes <christian@cheimes.de> Fri, 26 Jul 2013 22:45:00 +0200 https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/61b6cd7b9819/Modules/_pickle.c Christian Heimes <christian@cheimes.de> Mon, 01 Jul 2013 15:23:39 +0200 https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/17b7af660f82/Modules/_pickle.c segfault inside the _pickle C extension.]]> Christian Heimes <christian@cheimes.de> Mon, 01 Jul 2013 15:17:45 +0200 https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/54a44184bada/Modules/_pickle.c Alexandre Vassalotti <alexandre@peadrop.com> Sat, 20 Apr 2013 21:28:21 -0700 https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/4e412cbaaf96/Modules/_pickle.c Antoine Pitrou <solipsis@pitrou.net> Mon, 15 Apr 2013 21:51:09 +0200 https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/aa6f41599ef2/Modules/_pickle.c Alexandre Vassalotti <alexandre@peadrop.com> Sun, 14 Apr 2013 01:28:01 -0700 https://hg.dopieralski.pl/cpython-lru_cache-weakref/log/c1f846a99c85/Modules/_pickle.c Serhiy Storchaka <storchaka@gmail.com> Sat, 16 Feb 2013 17:29:56 +0200