cpython-lru_cache-weakref: Lib/dumbdbm.py annotate

annotate Lib/dumbdbm.py @ 79675:c0a2294de787 2.7

Issue #16199: Backport 2to3 typo fix from 3.2.

author	Chris Jerdonek <chris.jerdonek@gmail.com>
date	Thu, 11 Oct 2012 16:15:46 -0700
parents	5dae0d7f683a
children

rev	line source
2952 21df421f03fe Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	1 """A dumb and slow but simple dbm clone.
21df421f03fe Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	2
21df421f03fe Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	3 For database spam, spam.dir contains the index (a text file),
21df421f03fe Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	4 spam.bak may contain a backup of the index (also a text file),
21df421f03fe Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	5 while spam.dat contains the data (a binary file).
21df421f03fe Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	6
21df421f03fe Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	7 XXX TO DO:
21df421f03fe Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	8
21df421f03fe Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	9 - seems to contain a bug when updating...
21df421f03fe Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	10
21df421f03fe Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	11 - reclaim free space (currently, space once occupied by deleted or expanded
21df421f03fe Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	12 items is never reused)
21df421f03fe Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	13
21df421f03fe Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	14 - support concurrent access (currently, if two processes take turns making
21df421f03fe Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	15 updates, they can mess up the index)
21df421f03fe Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	16
21df421f03fe Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	17 - support efficient access to large databases (currently, the whole index
21df421f03fe Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	18 is read when the database is opened, and some updates rewrite the whole index)
21df421f03fe Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	19
21df421f03fe Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	20 - support opening for read-only (flag = 'm')
21df421f03fe Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	21
21df421f03fe Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	22 """
21df421f03fe Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	23
18729 d51b2d933814 Replace __import__ with import as. Martin v. Löwis <martin@v.loewis.de> parents: 17665 diff changeset	24 import os as _os
2952 21df421f03fe Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	25 import __builtin__
29628 bc1215f7dba3 SF 662923: iterator for dbm keys Raymond Hettinger <python@rcn.com> parents: 28125 diff changeset	26 import UserDict
2952 21df421f03fe Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	27
21df421f03fe Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	28 _open = __builtin__.open
21df421f03fe Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	29
21df421f03fe Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	30 _BLOCKSIZE = 512
21df421f03fe Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	31
15867 f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	32 error = IOError # For anydbm
3631 985134375672 export error == IOError Guido van Rossum <guido@python.org> parents: 3358 diff changeset	33
29628 bc1215f7dba3 SF 662923: iterator for dbm keys Raymond Hettinger <python@rcn.com> parents: 28125 diff changeset	34 class _Database(UserDict.DictMixin):
2952 21df421f03fe Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	35
29834 778b21520db8 Fixed critical shutdown race in _Database._commit. Tim Peters <tim.peters@gmail.com> parents: 29832 diff changeset	36 # The on-disk directory and data files can remain in mutually
778b21520db8 Fixed critical shutdown race in _Database._commit. Tim Peters <tim.peters@gmail.com> parents: 29832 diff changeset	37 # inconsistent states for an arbitrarily long time (see comments
778b21520db8 Fixed critical shutdown race in _Database._commit. Tim Peters <tim.peters@gmail.com> parents: 29832 diff changeset	38 # at the end of __setitem__). This is only repaired when _commit()
778b21520db8 Fixed critical shutdown race in _Database._commit. Tim Peters <tim.peters@gmail.com> parents: 29832 diff changeset	39 # gets called. One place _commit() gets called is from __del__(),
778b21520db8 Fixed critical shutdown race in _Database._commit. Tim Peters <tim.peters@gmail.com> parents: 29832 diff changeset	40 # and if that occurs at program shutdown time, module globals may
778b21520db8 Fixed critical shutdown race in _Database._commit. Tim Peters <tim.peters@gmail.com> parents: 29832 diff changeset	41 # already have gotten rebound to None. Since it's crucial that
29835 8d45a4cea934 Repaired typos in comments. Tim Peters <tim.peters@gmail.com> parents: 29834 diff changeset	42 # _commit() finish successfully, we can't ignore shutdown races
29834 778b21520db8 Fixed critical shutdown race in _Database._commit. Tim Peters <tim.peters@gmail.com> parents: 29832 diff changeset	43 # here, and _commit() must not reference any globals.
778b21520db8 Fixed critical shutdown race in _Database._commit. Tim Peters <tim.peters@gmail.com> parents: 29832 diff changeset	44 _os = _os # for _commit()
778b21520db8 Fixed critical shutdown race in _Database._commit. Tim Peters <tim.peters@gmail.com> parents: 29832 diff changeset	45 _open = _open # for _commit()
778b21520db8 Fixed critical shutdown race in _Database._commit. Tim Peters <tim.peters@gmail.com> parents: 29832 diff changeset	46
29827 2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	47 def __init__(self, filebasename, mode):
21301 12c094a23273 Honor the mode argument to dumbdbm.open(); there is not good reason not to, Fred Drake <fdrake@acm.org> parents: 20654 diff changeset	48 self._mode = mode
29827 2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	49
2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	50 # The directory file is a text file. Each line looks like
2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	51 # "%r, (%d, %d)\n" % (key, pos, siz)
2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	52 # where key is the string key, pos is the offset into the dat
2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	53 # file of the associated value's first byte, and siz is the number
2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	54 # of bytes in the associated value.
2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	55 self._dirfile = filebasename + _os.extsep + 'dir'
2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	56
2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	57 # The data file is a binary file pointed into by the directory
2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	58 # file, and holds the values associated with keys. Each value
2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	59 # begins at a _BLOCKSIZE-aligned byte offset, and is a raw
2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	60 # binary 8-bit string value.
2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	61 self._datfile = filebasename + _os.extsep + 'dat'
2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	62 self._bakfile = filebasename + _os.extsep + 'bak'
2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	63
2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	64 # The index is an in-memory dict, mirroring the directory file.
2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	65 self._index = None # maps keys to (pos, siz) pairs
2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	66
15867 f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	67 # Mod by Jack: create data file if needed
f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	68 try:
f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	69 f = _open(self._datfile, 'r')
f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	70 except IOError:
40509 2f84b6e81902 [Bug #802128] Make the mode argument of dumbdbm actually work the way it's Andrew M. Kuchling <amk@amk.ca> parents: 35038 diff changeset	71 f = _open(self._datfile, 'w')
2f84b6e81902 [Bug #802128] Make the mode argument of dumbdbm actually work the way it's Andrew M. Kuchling <amk@amk.ca> parents: 35038 diff changeset	72 self._chmod(self._datfile)
15867 f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	73 f.close()
f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	74 self._update()
2952 21df421f03fe Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	75
29827 2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	76 # Read directory file into the in-memory index dict.
15867 f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	77 def _update(self):
f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	78 self._index = {}
f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	79 try:
f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	80 f = _open(self._dirfile)
f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	81 except IOError:
f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	82 pass
f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	83 else:
29827 2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	84 for line in f:
35038 087735f86701 [Bug #1172763] dumbdbm uses eval() on lines, so it chokes if there's an extra \r on the end of a line; fixed by stripping off trailing whitespace. Andrew M. Kuchling <amk@amk.ca> parents: 29855 diff changeset	85 line = line.rstrip()
29827 2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	86 key, pos_and_siz_pair = eval(line)
2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	87 self._index[key] = pos_and_siz_pair
15867 f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	88 f.close()
f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	89
29827 2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	90 # Write the index dict to the directory file. The original directory
2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	91 # file (if any) is renamed with a .bak extension first. If a .bak
2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	92 # file currently exists, it's deleted.
15867 f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	93 def _commit(self):
29834 778b21520db8 Fixed critical shutdown race in _Database._commit. Tim Peters <tim.peters@gmail.com> parents: 29832 diff changeset	94 # CAUTION: It's vital that _commit() succeed, and _commit() can
778b21520db8 Fixed critical shutdown race in _Database._commit. Tim Peters <tim.peters@gmail.com> parents: 29832 diff changeset	95 # be called from __del__(). Therefore we must never reference a
778b21520db8 Fixed critical shutdown race in _Database._commit. Tim Peters <tim.peters@gmail.com> parents: 29832 diff changeset	96 # global in this routine.
29848 7052c3e5af02 Make close() identical to __del__() for a dumbdbm database. Make Tim Peters <tim.peters@gmail.com> parents: 29835 diff changeset	97 if self._index is None:
7052c3e5af02 Make close() identical to __del__() for a dumbdbm database. Make Tim Peters <tim.peters@gmail.com> parents: 29835 diff changeset	98 return # nothing to do
7052c3e5af02 Make close() identical to __del__() for a dumbdbm database. Make Tim Peters <tim.peters@gmail.com> parents: 29835 diff changeset	99
29827 2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	100 try:
29834 778b21520db8 Fixed critical shutdown race in _Database._commit. Tim Peters <tim.peters@gmail.com> parents: 29832 diff changeset	101 self._os.unlink(self._bakfile)
778b21520db8 Fixed critical shutdown race in _Database._commit. Tim Peters <tim.peters@gmail.com> parents: 29832 diff changeset	102 except self._os.error:
29827 2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	103 pass
2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	104
2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	105 try:
29834 778b21520db8 Fixed critical shutdown race in _Database._commit. Tim Peters <tim.peters@gmail.com> parents: 29832 diff changeset	106 self._os.rename(self._dirfile, self._bakfile)
778b21520db8 Fixed critical shutdown race in _Database._commit. Tim Peters <tim.peters@gmail.com> parents: 29832 diff changeset	107 except self._os.error:
29827 2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	108 pass
2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	109
40509 2f84b6e81902 [Bug #802128] Make the mode argument of dumbdbm actually work the way it's Andrew M. Kuchling <amk@amk.ca> parents: 35038 diff changeset	110 f = self._open(self._dirfile, 'w')
2f84b6e81902 [Bug #802128] Make the mode argument of dumbdbm actually work the way it's Andrew M. Kuchling <amk@amk.ca> parents: 35038 diff changeset	111 self._chmod(self._dirfile)
29828 5e81d92bf67d _commit(): Modernization. Tim Peters <tim.peters@gmail.com> parents: 29827 diff changeset	112 for key, pos_and_siz_pair in self._index.iteritems():
5e81d92bf67d _commit(): Modernization. Tim Peters <tim.peters@gmail.com> parents: 29827 diff changeset	113 f.write("%r, %r\n" % (key, pos_and_siz_pair))
15867 f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	114 f.close()
f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	115
29855 421d805c11f3 Give dumbdbm a sync() method which Shelve can call. Should solve some Skip Montanaro <skip@pobox.com> parents: 29848 diff changeset	116 sync = _commit
421d805c11f3 Give dumbdbm a sync() method which Shelve can call. Should solve some Skip Montanaro <skip@pobox.com> parents: 29848 diff changeset	117
15867 f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	118 def __getitem__(self, key):
f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	119 pos, siz = self._index[key] # may raise KeyError
f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	120 f = _open(self._datfile, 'rb')
f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	121 f.seek(pos)
f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	122 dat = f.read(siz)
f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	123 f.close()
f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	124 return dat
f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	125
29827 2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	126 # Append val to the data file, starting at a _BLOCKSIZE-aligned
2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	127 # offset. The data file is first padded with NUL bytes (if needed)
2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	128 # to get to an aligned offset. Return pair
2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	129 # (starting offset of val, len(val))
15867 f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	130 def _addval(self, val):
f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	131 f = _open(self._datfile, 'rb+')
f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	132 f.seek(0, 2)
f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	133 pos = int(f.tell())
19694 3716b116fb31 The first batch of changes recommended by the fixdiv tool. These are Guido van Rossum <guido@python.org> parents: 18729 diff changeset	134 npos = ((pos + _BLOCKSIZE - 1) // _BLOCKSIZE) * _BLOCKSIZE
15867 f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	135 f.write('\0'*(npos-pos))
f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	136 pos = npos
f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	137 f.write(val)
f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	138 f.close()
f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	139 return (pos, len(val))
f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	140
29827 2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	141 # Write val to the data file, starting at offset pos. The caller
2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	142 # is responsible for ensuring that there's enough room starting at
2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	143 # pos to hold val, without overwriting some other value. Return
2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	144 # pair (pos, len(val)).
15867 f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	145 def _setval(self, pos, val):
f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	146 f = _open(self._datfile, 'rb+')
f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	147 f.seek(pos)
f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	148 f.write(val)
f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	149 f.close()
f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	150 return (pos, len(val))
f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	151
29827 2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	152 # key is a new key whose associated value starts in the data file
29832 b9827c05460f More comments about why not closing a dumddbm properly can be a disaster. Tim Peters <tim.peters@gmail.com> parents: 29828 diff changeset	153 # at offset pos and with length siz. Add an index record to
b9827c05460f More comments about why not closing a dumddbm properly can be a disaster. Tim Peters <tim.peters@gmail.com> parents: 29828 diff changeset	154 # the in-memory index dict, and append one to the directory file.
29827 2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	155 def _addkey(self, key, pos_and_siz_pair):
2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	156 self._index[key] = pos_and_siz_pair
40509 2f84b6e81902 [Bug #802128] Make the mode argument of dumbdbm actually work the way it's Andrew M. Kuchling <amk@amk.ca> parents: 35038 diff changeset	157 f = _open(self._dirfile, 'a')
2f84b6e81902 [Bug #802128] Make the mode argument of dumbdbm actually work the way it's Andrew M. Kuchling <amk@amk.ca> parents: 35038 diff changeset	158 self._chmod(self._dirfile)
29827 2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	159 f.write("%r, %r\n" % (key, pos_and_siz_pair))
15867 f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	160 f.close()
f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	161
f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	162 def __setitem__(self, key, val):
f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	163 if not type(key) == type('') == type(val):
f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	164 raise TypeError, "keys and values must be strings"
29827 2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	165 if key not in self._index:
2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	166 self._addkey(key, self._addval(val))
15867 f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	167 else:
29827 2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	168 # See whether the new value is small enough to fit in the
2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	169 # (padded) space currently occupied by the old value.
15867 f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	170 pos, siz = self._index[key]
29809 9916b59261d7 __setitem__: Use integer division for computing # of blocks. Tim Peters <tim.peters@gmail.com> parents: 29628 diff changeset	171 oldblocks = (siz + _BLOCKSIZE - 1) // _BLOCKSIZE
9916b59261d7 __setitem__: Use integer division for computing # of blocks. Tim Peters <tim.peters@gmail.com> parents: 29628 diff changeset	172 newblocks = (len(val) + _BLOCKSIZE - 1) // _BLOCKSIZE
15867 f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	173 if newblocks <= oldblocks:
29827 2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	174 self._index[key] = self._setval(pos, val)
15867 f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	175 else:
29827 2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	176 # The new value doesn't fit in the (padded) space used
2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	177 # by the old value. The blocks used by the old value are
2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	178 # forever lost.
2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	179 self._index[key] = self._addval(val)
2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	180
2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	181 # Note that _index may be out of synch with the directory
2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	182 # file now: _setval() and _addval() don't update the directory
29832 b9827c05460f More comments about why not closing a dumddbm properly can be a disaster. Tim Peters <tim.peters@gmail.com> parents: 29828 diff changeset	183 # file. This also means that the on-disk directory and data
b9827c05460f More comments about why not closing a dumddbm properly can be a disaster. Tim Peters <tim.peters@gmail.com> parents: 29828 diff changeset	184 # files are in a mutually inconsistent state, and they'll
b9827c05460f More comments about why not closing a dumddbm properly can be a disaster. Tim Peters <tim.peters@gmail.com> parents: 29828 diff changeset	185 # remain that way until _commit() is called. Note that this
b9827c05460f More comments about why not closing a dumddbm properly can be a disaster. Tim Peters <tim.peters@gmail.com> parents: 29828 diff changeset	186 # is a disaster (for the database) if the program crashes
b9827c05460f More comments about why not closing a dumddbm properly can be a disaster. Tim Peters <tim.peters@gmail.com> parents: 29828 diff changeset	187 # (so that _commit() never gets called).
15867 f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	188
f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	189 def __delitem__(self, key):
29827 2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	190 # The blocks used by the associated value are lost.
15867 f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	191 del self._index[key]
29827 2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	192 # XXX It's unclear why we do a _commit() here (the code always
2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	193 # XXX has, so I'm not changing it). _setitem__ doesn't try to
2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	194 # XXX keep the directory file in synch. Why should we? Or
2ec2f62087cb There's a persistent rumor on the spambayes mailing list that dumbdbm Tim Peters <tim.peters@gmail.com> parents: 29809 diff changeset	195 # XXX why shouldn't __setitem__?
15867 f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	196 self._commit()
f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	197
f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	198 def keys(self):
f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	199 return self._index.keys()
f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	200
f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	201 def has_key(self, key):
23621 87f40377cf54 SF 563203. Replaced 'has_key()' with 'in'. Raymond Hettinger <python@rcn.com> parents: 23565 diff changeset	202 return key in self._index
15867 f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	203
17665 4c29b04e4d6f Added support for .__contains__(), .__iter__(), .iterkeys(). Fred Drake <fdrake@acm.org> parents: 16949 diff changeset	204 def __contains__(self, key):
23621 87f40377cf54 SF 563203. Replaced 'has_key()' with 'in'. Raymond Hettinger <python@rcn.com> parents: 23565 diff changeset	205 return key in self._index
17665 4c29b04e4d6f Added support for .__contains__(), .__iter__(), .iterkeys(). Fred Drake <fdrake@acm.org> parents: 16949 diff changeset	206
4c29b04e4d6f Added support for .__contains__(), .__iter__(), .iterkeys(). Fred Drake <fdrake@acm.org> parents: 16949 diff changeset	207 def iterkeys(self):
4c29b04e4d6f Added support for .__contains__(), .__iter__(), .iterkeys(). Fred Drake <fdrake@acm.org> parents: 16949 diff changeset	208 return self._index.iterkeys()
4c29b04e4d6f Added support for .__contains__(), .__iter__(), .iterkeys(). Fred Drake <fdrake@acm.org> parents: 16949 diff changeset	209 __iter__ = iterkeys
4c29b04e4d6f Added support for .__contains__(), .__iter__(), .iterkeys(). Fred Drake <fdrake@acm.org> parents: 16949 diff changeset	210
15867 f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	211 def __len__(self):
f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	212 return len(self._index)
f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	213
f5a3d98a35bc Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 15504 diff changeset	214 def close(self):
21516 0825eb1dbeb8 forward-patch from release21-maint branch: Anthony Baxter <anthonybaxter@gmail.com> parents: 21301 diff changeset	215 self._commit()
29848 7052c3e5af02 Make close() identical to __del__() for a dumbdbm database. Make Tim Peters <tim.peters@gmail.com> parents: 29835 diff changeset	216 self._index = self._datfile = self._dirfile = self._bakfile = None
2952 21df421f03fe Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	217
29848 7052c3e5af02 Make close() identical to __del__() for a dumbdbm database. Make Tim Peters <tim.peters@gmail.com> parents: 29835 diff changeset	218 __del__ = close
21958 5ed782bdbb2a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 21516 diff changeset	219
40509 2f84b6e81902 [Bug #802128] Make the mode argument of dumbdbm actually work the way it's Andrew M. Kuchling <amk@amk.ca> parents: 35038 diff changeset	220 def _chmod (self, file):
2f84b6e81902 [Bug #802128] Make the mode argument of dumbdbm actually work the way it's Andrew M. Kuchling <amk@amk.ca> parents: 35038 diff changeset	221 if hasattr(self._os, 'chmod'):
2f84b6e81902 [Bug #802128] Make the mode argument of dumbdbm actually work the way it's Andrew M. Kuchling <amk@amk.ca> parents: 35038 diff changeset	222 self._os.chmod(file, self._mode)
21516 0825eb1dbeb8 forward-patch from release21-maint branch: Anthony Baxter <anthonybaxter@gmail.com> parents: 21301 diff changeset	223
2952 21df421f03fe Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	224
21301 12c094a23273 Honor the mode argument to dumbdbm.open(); there is not good reason not to, Fred Drake <fdrake@acm.org> parents: 20654 diff changeset	225 def open(file, flag=None, mode=0666):
23565 1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 21958 diff changeset	226 """Open the database file, filename, and return corresponding object.
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 21958 diff changeset	227
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 21958 diff changeset	228 The flag argument, used to control how the database is opened in the
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 21958 diff changeset	229 other DBM implementations, is ignored in the dumbdbm module; the
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 21958 diff changeset	230 database is always opened for update, and will be created if it does
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 21958 diff changeset	231 not exist.
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 21958 diff changeset	232
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 21958 diff changeset	233 The optional mode argument is the UNIX mode of the file, used only when
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 21958 diff changeset	234 the database has to be created. It defaults to octal code 0666 (and
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 21958 diff changeset	235 will be modified by the prevailing umask).
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 21958 diff changeset	236
1a67fbc7a066 Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). Raymond Hettinger <python@rcn.com> parents: 21958 diff changeset	237 """
28125 f469a6d077f1 Fix comment, mode really is used Neal Norwitz <nnorwitz@gmail.com> parents: 23621 diff changeset	238 # flag argument is currently ignored
40510 f926bbc2fd62 [Bug #802128 continued] Modify mode depending on the process umask. Andrew M. Kuchling <amk@amk.ca> parents: 40509 diff changeset	239
f926bbc2fd62 [Bug #802128 continued] Modify mode depending on the process umask. Andrew M. Kuchling <amk@amk.ca> parents: 40509 diff changeset	240 # Modify mode depending on the umask
f926bbc2fd62 [Bug #802128 continued] Modify mode depending on the process umask. Andrew M. Kuchling <amk@amk.ca> parents: 40509 diff changeset	241 try:
f926bbc2fd62 [Bug #802128 continued] Modify mode depending on the process umask. Andrew M. Kuchling <amk@amk.ca> parents: 40509 diff changeset	242 um = _os.umask(0)
f926bbc2fd62 [Bug #802128 continued] Modify mode depending on the process umask. Andrew M. Kuchling <amk@amk.ca> parents: 40509 diff changeset	243 _os.umask(um)
f926bbc2fd62 [Bug #802128 continued] Modify mode depending on the process umask. Andrew M. Kuchling <amk@amk.ca> parents: 40509 diff changeset	244 except AttributeError:
f926bbc2fd62 [Bug #802128 continued] Modify mode depending on the process umask. Andrew M. Kuchling <amk@amk.ca> parents: 40509 diff changeset	245 pass
f926bbc2fd62 [Bug #802128 continued] Modify mode depending on the process umask. Andrew M. Kuchling <amk@amk.ca> parents: 40509 diff changeset	246 else:
f926bbc2fd62 [Bug #802128 continued] Modify mode depending on the process umask. Andrew M. Kuchling <amk@amk.ca> parents: 40509 diff changeset	247 # Turn off any bits that are set in the umask
f926bbc2fd62 [Bug #802128 continued] Modify mode depending on the process umask. Andrew M. Kuchling <amk@amk.ca> parents: 40509 diff changeset	248 mode = mode & (~um)
40732 5dae0d7f683a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 40510 diff changeset	249
21301 12c094a23273 Honor the mode argument to dumbdbm.open(); there is not good reason not to, Fred Drake <fdrake@acm.org> parents: 20654 diff changeset	250 return _Database(file, mode)

Mercurial > cpython-lru_cache-weakref

annotate Lib/dumbdbm.py @ 79675:c0a2294de787 2.7