Re: Simple Web Storage

On Wed, Jul 1, 2009 at 3:35 PM, Chris Anderson <jchris@apache.org> wrote:

> On Wed, Jul 1, 2009 at 6:11 PM, Jeremy Orlow<jorlow@chromium.org> wrote:
> > On Wed, Jul 1, 2009 at 2:55 AM, Chris Anderson <jchris@apache.org>
> wrote:
> >>
> >> Hello,
> >>
> >> Long time fan, first time writer. ;)
> >>
> >> I've been following the Web Storage proposals with interest, and was
> >> just independently drafting a mail suggesting the a B-Tree API would
> >> be much simpler to standardize, and would be an adequate foundation
> >> for building most anything more specific.
> >>
> >> I'd like to express my support for a BDB style API. Firstly because it
> >> would be much less prone to vendor differences, and secondly because I
> >> can see how to implement a performant CouchDB on top of it. It could
> >> be a bit harder to build a SQL-like interface on raw B-Trees, but
> >> completely possible if someone is determined.
> >>
> >> I don't think we need to worry about specific vendor implementations,
> >> just a unified API with very small surface area. To build CouchDB all
> >> we'd need is B-Trees that support in-order and reverse-order
> >> traversal, and optionally user-defined collation functions.
> >>
> >> Here's a first cut at imagining the API:
> >>
> >> === JS pseudocode ===
> >>
> >> var btree = new WebStore("dbname", <optional collation function
> >> definition>);
> >>
> >> btree["mydocid"] = {"some":"json"};
> >>
> >> btree.forEach(function(key, value) {
> >>  // in order traversal
> >> })
> >>
> >> btree.forEach(function(key, value) {
> >>  // reverse order traversal
> >> }, false)
> >>
> >> btree.forEach("startkey", function(key, value) {
> >>  // in order traversal, starting from "startkey"
> >>  // we could use throw() to stop traversal
> >> })
> >>
> >> btree.forEach("endkey", function(key, value) {
> >>  // reverse order traversal, starting from "endkey"
> >>  // use throw() to stop traversal
> >> }, false)
> >>
> >> // delete a btree
> >> WebStore.drop("dbname");
> >>
> >> ===
> >>
> >> Thoughts?
> >
> > The biggest thing I see missing is transactions.
> >
> > Also, I don't know why web databases decided not to allow deletion or
> > enumeration of the database names, but I assume it was difficult or
> > impossible to do so in a non-racy way.  It's definitely worth looking
> into
> > this before supporting some sort of drop.  And if we did support it, it
> > seems as though the name could be more clear.
> >
> > The rest seems reasonable.
> >
> > I'm still concerned that this is not differentiated enough with
> > LocalStorage.  The differences are: duplicate keys, values besides
> strings,
> > multiple stores, more powerful enumeration, and transactions (maybe
> others
> > I'm missing?).  Yet, from a novice developers perspective, I don't think
> > it'd be clear at all what the differences are.  I'm not sure how to
> > reconcile this since LocalStorage is one of the standards supported by
> > pretty much everyone--including IE.
> >
>
> Maybe I should be looking at LocalStorage. All I need is in-order and
> reverse keyspace traversal and custom collation.


What do you mean by custom collation?

You're not going to get in order traversal from LocalStorage as is.  But if
something like the "b-tree API" doesn't move forward, it might be worth
adding to the LocalStorage API.


> I agree an "all_dbs" command would be useful.
>
> I should benchmark some of the LocalStorage modules, maybe they are
> fast enough, and large enough (Gigabytes?) Guess I'll find out.


I can tell you WebKit's (and soon Chromium's) implementation isn't designed
with this in mind...but part of the reason is that no one's pushing this to
its limits yet.  :-)


> Multi-key transactions can get tricky in distributed applications, so
> CouchDB doesn't use them, but if they make sense for other use cases,
> then we'll still be free not to use them.


Yeah.  I think the database API's handling of transactions might be a good
place to start.

LocalStorage gets around this by having per-origin locks that are implicitly
taken on first access to LocalStorage and which are dropped either when you
exit the JavaScript context, call navigator.unlockStorage(), or do something
synchronous....but I think we should avoid doing that with any new APIs.

J