Re: IndexedDB, what were the issues? How do we stop it from happening again?

On Wednesday, March 6, 2013, Ian Fette (イアンフェッティ) wrote:

> I seem to recall we contemplated people writing libraries on top of IDB
> from the beginning. I'm not sure why this is a bad thing.
>

It's not bad as an assumption, but it can quickly turn into an excuse for
API design malpractice because it often leads to the (mistaken) assumption
that user-provided code is as cheap as browser-provided API. Given that
users pull their libraries from the network more often than from disk (and
must parse/compile, etc.), the incentives of these two API-providers could
not be more different. That's why it's critical that API designers try to
forestall the need for libraries for as long as possible when it comes to
web features.


> We originally shipped "web sql" / sqlite, which was a familiar interface
> for many and relatively easy to use, but had a sufficiently large API
> surface area that no one felt they wanted to document the whole thing such
> that we could have an inter-operable standard. (Yes, I'm simplifying a bit.)
>

Yeah, I recall that the SQLite semantics were the big obstacle.


> As a result, we came up with an approach of "What are the fundamental
> primitives that we need?", spec'd that out, and shipped it. We had
> discussions at the time that we expected library authors to produce
> abstraction layers that made IDB easier to use, as the "fundamental
> primitives" approach was not necessarily intended to produce an API that
> was as straightforward and easy to use as what we were trying to replace.
> If that's now what is happening, that seems like a good thing, not a
> failure.
>

It's fine in the short run to provide just the low-level stuff and work up
to the high-level things -- but only when you can't predict what the
high-level needs will be. Assuming that's what the WG's view was, you're
right; feature not bug, although there's now more work to do.

Anyhow, IDB is incredibly high-level in many places and primitive in
others. ISTM that it's not easy to get a handle on it's intended level of
abstraction.


> On Wed, Mar 6, 2013 at 10:14 AM, Alec Flett <alecflett@chromium.org>wrote:
>
> My primary takeaway from both working on IDB and working with IDB for some
> demo apps is that IDB has just the right amount of complexity for really
> large, robust database use.. but for a "welcome to noSQL in the browser" it
> is way too complicated.
>
> Specifically:
>
>    1. *versioning* - The reason this exists in IDB is to guarantee a
>    schema (read: a fixed set of objectStores + indexes) for a given set of
>    operations.  Versioning should be optional. And if versioning is optional,
>    so should *opening* - the only reason you need to "open" a database is
>    so that you have a handle to a versioned database. You can *almost* implement
>    versioning in JS if you really care about it...(either keep an explicit
>    key, or auto-detect the state of the schema) its one of those cases where
>    80% of versioning is dirt simple  and the complicated stuff is really about
>    maintaining version changes across multiply-opened windows. (i.e. one
>    window opens an idb, the next window opens it and changes the schema, the
>    first window *may* need to know that and be able to adapt without
>    breaking any in-flight transactions) -
>    2. *transactions* - Also should be optional. Vital to complex apps,
>    but totally not necessary for many.. there should be a default transaction,
>    like db.objectStore("foo").get("bar")
>    3. *transaction scoping* - even when you do want transactions, the api
>    is just too verbose and repetitive for "get one key from one object store"
>    - db.transaction("foo").objectStore("foo").get("bar") - there should be
>    implicit (lightweight) transactions like db.objectStore("foo").get("bar")
>    4. *forced versioning* - when versioning is optional, it should be
>    then possible to change the schema during a regular transaction. Yes, this
>    is a lot of rope but this is actually for much more complex apps, rather
>    than simple ones. In particular, it's not uncommon for more complex
>    database systems to dynamically create indexes based on observed behavior
>    of the API, or observed data (i.e. when data with a particular key becomes
>    prevalent, generate an index for it) and then dynamically use them if
>    present. At the moment you have to do a manual close/open/version change to
>    dynamically bump up the version - effectively rendering fixed-value
>    versions moot (i.e. the schema for version 23 in my browser may look
>    totally different than the schema for version 23 in your browser) and
>    drastically complicating all your code (Because if you try to close/open
>    while transactions are in flight, they will be aborted - so you have to
>    temporarily pause all new transactions, wait for all in-flight transactions
>    to finish, do a close/open, then start running all pending/paused
>    transactions.) This last case MIGHT be as simple as adding
>    db.reopen(newVersion) to the existing spec.
>    5. *named object stores* - frankly, for *many* use cases, a single
>    objectStore is all you need. a simple db.get("foo") would be sufficient.
>    Simply naming a "default" isn't bad - whats bad is all the onupgradeneeded
>    scaffolding required to create the objectstore in the first place.
>
> I do think that the IDBRequest model needs tweaking, and Futures seem like
> the obvious direction to head in.
>
> FWIW, the "sync" version of the API is more or less dead - nobody has
> actually implemented it.
>
> I think there is a very specialized set of applications that absolutely
> need the features that IDB has right now. Google Docs is a perfect example
> - long lived complicated application that needs to keep absolute integrity
> of schema across multiple tabs over a long period of time.. but for 99% of
> usecases out there, I think they're unnecessary.
>
> I think ultimately, a simplified IDB would allow progressive use of the
> api as your application grows.
>
> // basic interaction - some objectStore named 'default' gets crated under
> the hood.
> indexedDB.get("mykey");
> // named database, auto-create the 'first' objectStore named 'default', no
> need to 'close' anything
> indexedDB.database("mydb").get("mykey")
> // now we need multiple objectstores:
> indexedDB.database("mydb").objectStore("default").get("mykey")
> // time for versioning, but using 'default'
> indexedDB.open("mydb", 12).onupgrad
>
>