Yehuda Katz is a member of the Ember.js, Ruby on Rails and jQuery Core Teams; he spends his daytime hours at the startup he founded, Tilde Inc.. Yehuda is co-author of best-selling jQuery in Action and Rails 3 in Action. He spends most of his time hacking on open source—his main projects, like Thor, Handlebars and Janus—or traveling the world doing evangelism work. He can be found on Twitter as @wycats and on Github.

Initial Release of Moneta: Unified Key/Value Store API

I’m happy to announce the (very) initial release of a new library called moneta, which aims to provide a unified interface for key/value stores. The interface mirrors the key/value API of hash, but not the iteration API, since iteration is not possible across all key/value stores. It is likely that a future release will add support for enumeration by keeping a list of keys in a key (very meta :P )

In order to prove out the API, I created five highly experimental stores for the initial release:

  • A basic file store: It uses the file name as the key, the file contents as the value, and xattrs to hold expiration information. The file store requires the xattr gem
  • An xattrs-only store: This uses a single file’s xattrs for keys and values, and a second file’s xattrs to hold an identical hash with expiration information
  • An in-memory store: This was the first store I wrote, purely to prove out the API. It uses a Hash internally, and a second hash for expiration information
  • A DataMapper store: Uses any DataMapper-supported storage with three columns: the_key, value, and expiration
  • A memcached store: Uses memcached and its native expiration

Note that the initial release also does not do any kind of locking; a Moneta Store should be treated as a standard hash, and should be locked appropriately by the consumer. Also note that the stores themselves do not perform any locking, so they should probably not be used between processes at this time (i.e. they are only experimental). It should be pretty straight-forward to add locking to most of the stores.

Likely the only locking issue in the memcached store is in #delete, which does:

def delete(key)
  value = self[key]
  @cache.delete(key) if value
  value
end

As a result, it is possible (at the moment, again) for delete to return a key that has been modified before it was actually deleted. For most use-cases, this is rather unlikely to matter, but keep in mind that pretty much all of the stores are unoptimized and not concurrent-safe across processes. Treat the stores, at the moment, as proofs of concept for the overall API.

Some details

The Moneta API is purposely extremely similar to the Hash API. In order so support an
identical API across stores, it does not support iteration or partial matches, but that
might come in a future release.

The API:

#initialize(options)
options differs per-store, and is used to set up the store

#[](key)
retrieve a key. if the key is not available, return nil

#[]=(key, value)
set a value for a key. if the key is already used, clobber it.
keys set using []= will never expire

#delete(key)
delete the key from the store and return the current value

#key?(key)
true if the key exists, false if it does not

#has_key?(key)
alias for key?

#store(key, value, options)
same as []=, but you can supply an :expires_in option, 
which will specify a number of seconds before the key
should expire. In order to support the same features
across all stores, only full seconds are supported

#update_key(key, options)
updates an existing key with a new :expires_in option.
if the key has already expired, it will not be updated.

#clear
clear all keys in this store

15 Responses to “Initial Release of Moneta: Unified Key/Value Store API”

I’d like to know *what* the motivation and purpose behind this library was/is.

That’s exactly what jmettraux has been doing with Tokyo Cabinet for rufus-tokyo on github.

Of course, Tokyo Cabinet has Tyrant (network access) and Dystopia (indexed search).

Awesome API though, can’t wait to work with it!

@luke every frakking framework out there reimplements this stuff, and I wanted to make it easy for the community to share stores and for frameworks to just use them. Kind of like DO and Rack.

I was also chatting with dkubb (maintainer of DM) last night and there were some interesting possibilities for making the IdentityMap of DataMapper support any moneta store.

The bottom line is that the more of these pluggable APIs there are, the more possible it is to build powerful combinations of the APIs.

How do I install this on my Leopard system? (that’s a newbie question)

I want to try it out using irb.

Try: sudo gem install wycats-moneta --source http://gems.github.com.

To use the file or xattr store, you will need the xattr gem. To use the DataMapper store you will need the DataMapper gem. To use the memcache store you will need the memcache-client gem.

To use a store once installed, simply require "moneta/memcache" or the name of the store. You can see how to instantiate the stores in the specs.

FYI: If you need to coordinate locking between processes, you should consider using elock: http://dustin.github.com/elock/

Will add support for moneta to Ramaze.
I can contribute support for a Sequel and Og interface as well as maybe one for CouchDB and MongoDB.
/me goes off forking

Hey, Yehuda, I bet your page is not supposed to say this:

Warning: fread() [function.fread]: Length parameter must be greater than 0. in /home/2424/domains/yehudakatz.com/html/wp-content/themes/yehuda/functions.php on line 12

Appeared in my page load under the Twitter logo in the header.

Yehuda, Thanks for your hard work and development. We all benefit from your efforts.

Ii am looking for help with Moneta. I can’t seem to find a forum anywhere, so I am going to try here.

I am trying to use the File option on Moneta. On my development mac, everything is peachy, but when I went to run my test on my deployment server, which is a Joyent solaris box, I ran into trouble.

I installed the moneta gem and the xattr gem as indicated, but everytime I try to create a new filestore, I get errors. This particular example comes from trying to build the filestore in the console, but the results are the same regardless of whether they are coming from my tests, my code, or what directory path I pass to the :path option.

>> m = Moneta::File.new(:path => “filestore”)

>> m = Moneta::File.new(:path => “filestore”)

Leave a Reply

Archives

Categories

Meta