odbm = object wrapper for key-value stores

odbm

Project optimization => storage optimization => dbm => dict style => :-( => odmb => :-)


')

Backends

• tokyo cabinet (via tokyo-python ) - I strongly recommend, a couple of times faster than analogs, the database file is more compact
• kyoto cabinet (through the native wrapper ) - from the authors of tokyo-cabinet and the description of them is steeper, but according to my tests it’s 2-3 times slower
• gdbm
• fsdbm - convenient for storing large documents
• ... write if something is missing
  
  

  Installation

  
  pip install odbm

  turnip: bitbucket
  
  

  Syntax

  
  
  

   from datetime import datetime, date import odbm #   class User(odbm.Model): username = odbm.UnicodeProperty(primary_key=True) birthday = odbm.DateProperty(key='b') is_active = odbm.Property(key='a', default=True) is_admin = odbm.Property(key='r', default=False) created = odbm.DateTimeProperty(key='c') __filename__ = 'var/user.odbm.tch' __db_type__ = 'tc' #     User(username='fizban', birthday=date(1917, 1, 1), created=datetime.now()).save() #  dict-style  : # db['fizban'] = {'b': timestamp , 'c': timestamp } #      (. key=...  property) #     u = User.get('fizban') print u.birthday # datetime.date(1917, 1, 1) #   u.is_active = False u.save() #    for i in xrange(10): User(username=('test-%i' % i), birthday=date(1950 + i, 1, 1)).save() #   User.find() #    c birthday > 1955,   username for obj in User.find( filter = lambda u: u.birthday.year > 1955, order = lambda u: u.username): print obj #     print User.count(lambda u: u.username.startswith('f')) # 1 #   print User.count() # 11 User.find_one().delete() print User.count() # 10 

  
  
  

  Property

  
  
  • odbm.Property () - any marshall types can be pushed here (for example, {'a': [True, 1, None]})
  • odbm.UnicodeProperty () - for unicode strings to eliminate confusion with Latin ascii and unicode keys
  • odbm.DateTimeProperty () - datetime.datetime
  • odbm.DateProperty () - datetime.date
  • odbm.CompressedProperty () - like odbm.Property, but compressed
  • odbm.CompressedUnicodeProperty - like odbm.UnicodeProperty, but compressed
  
  
  

  Performance

  
  For the test, we write and count in random order 10k Users of the above model. And compare with writing / reading the same data in the form of cPickle dictionaries:
  
  seconds to write/read 10000 rows
pickle (write, read): (1.0030829906463623, 0.25429201126098633)
odbm (write, read): (0.99477910995483398, 0.14065694808959961)
pickle file size: 2768704
odbm file size: 1348752

  
  Reducing the file size is achieved by reducing the key length described above. Test code here .
  
  

  [offtop] On the fundamental importance of the compactness of the database file

  
  
  A year ago, we conducted a series of tests that clearly showed: regardless of the subd engine, as soon as the files used are not placed in the file system cache, the speed of random samples falls by a couple of orders of magnitude . If someone is interested in the numbers, that's all that remains of those tests:
  
  • dumpz.org/19177
  • dumpz.org/19243
  • dumpz.org/19255