[IndexedDB] Design Flaws: Not Stateless, Not Treating Objects As Opaque

> On 20 Mar 2011, at 4:54 AM, Jonas Sicking wrote:
> 
> I don't understand what you are saying about application state though,
> so please do start that as a separate thread.

At present, there's no way for an application to tell IDB what indexes to modify w.r.t. an object at the exact moment when putting or deleting that object. That's because this behavior is defined in advance using "createIndex" in a "setVersion" transaction. And then how IDB extracts the referenced value from the object is done using an IDB idea of "key paths". But right there, in defining the indexes in advance (and not when the index is actually modified, which is when the object itself is modified), you've captured application state (data relationships that should be known only to the application) within IDB. Because this is done in advance (because IDB seems to have inherited this assumption that this is just the way MySQL happens to do it), there's a disconnect between when the index is defined and when it's actually used. And because of "key paths" you now need to spec out all kinds of things like how to handle compound keys, multiple values. It's becoming a bit of a spec-fest.

That this bubble of state gets captured in IDB, it also means that IDB now needs to provide ways of updating that captured state within IDB when it changes in the application (which will happen, so essentially you now have your indexing logic stuck in the database AND in the application and the application developer now has to try and keep BOTH in sync using this awkward pre-defined indexes interface), thus the need for a setVersion transaction in the first place. None of this would be necessary if the application could reference indexes to be modified (and created if they don't exist, or deleted if they would then become empty) AT THE POINT of putting or deleting an object. Things like data migrations would also be better served if this were possible since this is something the application would need to manage anyway. Do you follow?

The application is the right place to be handling indexing logic. IDB just needs to provide an interface to the indexing implementation, but not handle extracting values from objects or deciding which indexes to modify. That's the domain of the application. It's a question of encapsulation. IDB is crossing the boundaries by demanding to know ABOUT the data stored, and not just providing a simple way to put an object, and a simple way to put a reference to an object to an index, and a simple way to query an index and intersect or union an index with another. Essentially an object and its index memberships need to be completely opaque to IDB and you are doing the opposite. Take a look at the BDB interface. Do you see a setVersion or createIndex semantic in there? Take a look at Redis and Tokyo and many other things. Do you see a setVersion or createIndex semantic in there? Do these databases have any idea about the contents of objects? Any concept of key paths? No, and that's the whole reason these databases were created in the first place. I'm sure you have read the BDB papers. Obviously this is not the approach of MySQL. But if IDB is trying to be MySQL but saying it wants to be BDB then I don't know. In any event, Firefox would be brave to also embed SQLite. Let the better API win.

How much simpler could it be? At the end of the day, it's all objects and sets and sorted sets, and see Redis' epiphany on this point. IDB just needs to provide transactional access to these sets. The application must decide what goes in and out of these sets, and must be able to do it when it wants to, not some time in advance. I bring this up because I once wrote the exact same kind of database that you are writing now (where one thinks it would be good if the database did NOT treat objects as opaque... that the database should be smart about the contents of objects and share control for how objects relate to each other etc.) and I have since seen how much better, simpler, faster the alternative is. So unless you have formidable reasons for maintaining the status quo in light of the above, even if you don't understand this concept of application state getting stuck in IDB, and even though you advocate that WebSQL is not deprecated and that we can consider LocalStorage to be an alternative, then it is my hope that you will heed this and make something of it. I'm sorry if this is not the kind of feedback you want to hear at this stage, but IDB needs to be good for more than just HTML 5 todo list demos.

Received on Sunday, 20 March 2011 22:11:19 UTC