Re: Replacing WebSQL with a Relational Data Model.

Hi,

I will comment inline:

On 26 October 2010 09:42, Jonas Sicking <jonas@sicking.cc> wrote:

> On Tue, Oct 26, 2010 at 12:02 AM, Keean Schupke <keean@fry-it.com> wrote:
> > Hi,
> > It will be a lot faster with SQLite as the backend. Mobile apps depend on
> > access to the SQLite engine, and although it _could_ be implemented on
> top
> > of IndexedDB, there is no way its going to be fast enough... And the
> thought
> > of writing a decent query optimiser is a bit daunting.
> > The benefit of making it a standard, is that browser implementers can do
> > what they want, implement in JavaScript on top of IndexedDB, or pass
> through
> > to a proper database.
> > BTree's are old hat in the database space. Modern storage engines are
> using
> > Fractal-Trees for a 50-80 times speedup in write performance. The browser
> > should not be trying to compete with experienced database coders.
> > The relational storage model I am proposing will return a database table
> as
> > a list of objects, where each object represents a row, and the object has
> a
> > set of named properties representing each column.
> > Part of the power of the Relational Data Model is that it abstracts data
> > into columns and tables, and this is precisely what we want.
>
> I think you are misunderstanding what I'm proposing.
>

This is a definite possibility... My intelligence is borrowed from other
people, my stupidity is my own...


>
> The problem is two-fold. I don't think we want to create a entirely
> new database backend. We already have two different APIs (three if you
> count localStorage), and none of them have full buy-in from all major
> browser vendors. Adding a third one, and on top of that one that is
> more complex than all other ones that have been suggested, I don't
> think would be successful.
>

Backend? Frontend? Are you suggesting that IndexedDB could be implemented on
top of an SQL database?


>
> However I do think that what you are proposing could be done as
> additional API for the IndexedDB spec. However since we are currently
> trying to finish up the currently discussed feature set, I don't think
> we want to take on what you are proposing *at this time*. I do however
> think that it's something we should look into for the next version of
> IndexedDB.
>

Which version is not important. I guess what I am looking for is a
willingness from the implementers to provide a relational API, backed by a
fast database (like SQLite).


>
> In the *meantime*, what you could do is build *prototypes* in
> javascript on top of IndexedDB. This would have several benefits:
>
> * It would be easier for everyone to understand exactly what is being
> proposed as we could look at your prototype implementation.
>

Totally agree.


> * It would give web authors something to experiment with so that they
> can say if they like it or not. Or if they would like something
> changed.
>

Ditto.


> * It would allow us to profile to see which parts are slow and so that
> we can prioritize those areas first for specification in IndexedDB v2
> and native implementation in browsers.
>

Here is where there is some miscommunication (likely to be mine). What I am
proposing is a relational API, that is not an implementation of the RDBMS
itself. Although you could directly implement a relational algebra in
JavaScript, this would be a lot of work.

The way to view this is to think of a reverse-parser. You might construct a
query something like:

q1 = rdb.table(myTable);
q2 = q1.restrict(rdb.eq(function(row) {row[myColumn] == myValue}));
result_table = rdb.query(q2);

This would construct an abstract syntax tree. So q1 is a tree with a single
node representing the table "myTable". q2 is a tree with a root that is a
"restrict" node representing the equality (itself an expression tree) and a
child (the original q1 node).

When we execute this by passing it to "rdb.query" the syntax tree is
"rendered" to a string in this case we would produce:

"select * from myTable where myColumn = 'myValue';"

This is then passed through to the RDBMS to execute, and the table returned
in marshalled into the correct format.

So to implement this on top of WebSQL would be a the plan for a prototype.
To implement a complete relational database engine on top of IndexedDB would
be hard. To get good performance would be even harder.


>
> So I'm not saying that IndexedDB in its current state is enough to
> build what you want and get good performance. I'm saying that I think
> a prototype implementation on top of IndexedDB would be very
> educational.
>

Implementing the RDBMS would be hard. Implementing the API on-top of an
external RDBMS (SQLite) would be the way to go. Could this still be part of
IndexedDB?


>
> Of course, if you disagree and think writing a separate database
> interface is the better way to go, then don't let me stop you. You are
> totally free to write a specification and I'd imagine W3C would be
> happy to publish it. I'd think it would fit under our current charter
> and so very little red tape is needed.
>

I would prefer to fit in with what everyone is already doing. There would be
no point in starting a new standard if none of the browser implementers are
interested in this. I would also prefer to have the cooperation of an
experiences W3 standards editor if this is necessary.


Cheers,
Keean.