Yehuda Katz is a member of the Ember.js, Ruby on Rails and jQuery Core Teams; his 9-to-5 home is at the startup he founded, Tilde Inc.. There he works on Skylight, the smart profiler for Rails, and does Ember.js consulting. He is best known for his open source work, which also includes Thor and Handlebars. He travels the world doing open source evangelism and web standards work.

Initial Release of Moneta: Unified Key/Value Store API

I’m happy to announce the (very) initial release of a new library called moneta, which aims to provide a unified interface for key/value stores. The interface mirrors the key/value API of hash, but not the iteration API, since iteration is not possible across all key/value stores. It is likely that a future release will add support for enumeration by keeping a list of keys in a key (very meta :P )

In order to prove out the API, I created five highly experimental stores for the initial release:

  • A basic file store: It uses the file name as the key, the file contents as the value, and xattrs to hold expiration information. The file store requires the xattr gem
  • An xattrs-only store: This uses a single file’s xattrs for keys and values, and a second file’s xattrs to hold an identical hash with expiration information
  • An in-memory store: This was the first store I wrote, purely to prove out the API. It uses a Hash internally, and a second hash for expiration information
  • A DataMapper store: Uses any DataMapper-supported storage with three columns: the_key, value, and expiration
  • A memcached store: Uses memcached and its native expiration

Note that the initial release also does not do any kind of locking; a Moneta Store should be treated as a standard hash, and should be locked appropriately by the consumer. Also note that the stores themselves do not perform any locking, so they should probably not be used between processes at this time (i.e. they are only experimental). It should be pretty straight-forward to add locking to most of the stores.

Likely the only locking issue in the memcached store is in #delete, which does:

def delete(key)
  value = self[key]
  @cache.delete(key) if value

As a result, it is possible (at the moment, again) for delete to return a key that has been modified before it was actually deleted. For most use-cases, this is rather unlikely to matter, but keep in mind that pretty much all of the stores are unoptimized and not concurrent-safe across processes. Treat the stores, at the moment, as proofs of concept for the overall API.

Some details

The Moneta API is purposely extremely similar to the Hash API. In order so support an
identical API across stores, it does not support iteration or partial matches, but that
might come in a future release.

The API:

options differs per-store, and is used to set up the store

retrieve a key. if the key is not available, return nil

#[]=(key, value)
set a value for a key. if the key is already used, clobber it.
keys set using []= will never expire

delete the key from the store and return the current value

true if the key exists, false if it does not

alias for key?

#store(key, value, options)
same as []=, but you can supply an :expires_in option, 
which will specify a number of seconds before the key
should expire. In order to support the same features
across all stores, only full seconds are supported

#update_key(key, options)
updates an existing key with a new :expires_in option.
if the key has already expired, it will not be updated.

clear all keys in this store

15 Responses to “Initial Release of Moneta: Unified Key/Value Store API”

I’d like to know *what* the motivation and purpose behind this library was/is.

That’s exactly what jmettraux has been doing with Tokyo Cabinet for rufus-tokyo on github.

Of course, Tokyo Cabinet has Tyrant (network access) and Dystopia (indexed search).

Awesome API though, can’t wait to work with it!

@luke every frakking framework out there reimplements this stuff, and I wanted to make it easy for the community to share stores and for frameworks to just use them. Kind of like DO and Rack.

I was also chatting with dkubb (maintainer of DM) last night and there were some interesting possibilities for making the IdentityMap of DataMapper support any moneta store.

The bottom line is that the more of these pluggable APIs there are, the more possible it is to build powerful combinations of the APIs.

How do I install this on my Leopard system? (that’s a newbie question)

I want to try it out using irb.

Try: sudo gem install wycats-moneta --source

To use the file or xattr store, you will need the xattr gem. To use the DataMapper store you will need the DataMapper gem. To use the memcache store you will need the memcache-client gem.

To use a store once installed, simply require "moneta/memcache" or the name of the store. You can see how to instantiate the stores in the specs.

FYI: If you need to coordinate locking between processes, you should consider using elock:

Will add support for moneta to Ramaze.
I can contribute support for a Sequel and Og interface as well as maybe one for CouchDB and MongoDB.
/me goes off forking

Hey, Yehuda, I bet your page is not supposed to say this:

Warning: fread() [function.fread]: Length parameter must be greater than 0. in /home/2424/domains/ on line 12

Appeared in my page load under the Twitter logo in the header.

Yehuda, Thanks for your hard work and development. We all benefit from your efforts.

Ii am looking for help with Moneta. I can’t seem to find a forum anywhere, so I am going to try here.

I am trying to use the File option on Moneta. On my development mac, everything is peachy, but when I went to run my test on my deployment server, which is a Joyent solaris box, I ran into trouble.

I installed the moneta gem and the xattr gem as indicated, but everytime I try to create a new filestore, I get errors. This particular example comes from trying to build the filestore in the console, but the results are the same regardless of whether they are coming from my tests, my code, or what directory path I pass to the :path option.

>> m = => “filestore”)

>> m = => “filestore”)

Leave a Reply